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1.  SUMMARY 


This  report  documents  the  purpose,  background,  scope,  preparation,  and  execution  of  the 
Robotics  Range  Clearance  Competition  (R2C2)  sponsored  by  the  Joint  Ground  Robotics 
Enterprise  (JGRE),  the  US  Anny  Corp  of  Engineers  (USACE),  and  the  Air  Force  Research 
Laboratory  (AFRL).  The  Office  of  Secretary  of  Defense  (OSD)  in  collaboration  with  the  US  Air 
Force  and  US  Army  conducted  a  Robotic  Range  Clearance  Competition  (R2C2)  to  foster  the 
ability  to  clear  training  ranges  of  debris  and  unexploded  ordnance  (UXO)  using  robotic 
technologies.  The  purpose  of  the  Robotic  Range  Clearance  Competition  was  to  quickly  tap  into 
the  innovation  and  ingenuity  of  the  commercial  robotic  technology  sector  to  improve  the  safety 
and  effectiveness  of  the  four  tasks  traditionally  associated  with  range  clearing  operations:  1) 
Vegetation  Clearance,  2)  Surface  Clearance,  3)  Geophysical  Mapping,  and  4)  Subsurface 
Clearance. 

The  R2C2  Competition  teams  were: 

•  Team  UXOD  Automation  -  comprised  of  Kairos  Autonomi,  Autonomous  Solutions, 
SAIC,  Zonge  Engineering,  Vallon,  WM  Robots,  VKR,  Inc.,  and  John  Deere 

•  Team  D4C  -  comprised  of  ECC,  QinetiQ  North  America,  and  Bobcat 

•  Team  Sky  Research 

Planning  for  the  competition  began  in  2009  and  was  publicly  announced  with  a  kickoff  meeting 
and  briefings  in  Washington,  DC  in  October  2009.  The  competition  events  were  held  at  Camp 
Guernsey,  WY  from  Sunday,  7  August  through  Saturday,  13  August  2012.  Team  UXOD 
Automation  and  Team  D4C  both  competed  in  all  four  events.  Team  Sky  Research  only  competed 
in  the  geophysical  mapping  event. 

The  results  of  the  R2C2  were  announced  17  August  2012  at  the  Association  of  Unmanned 
Vehicle  Systems  International  (AUVSI)  North  America  2011  conference  in  Washington,  D.C. 
Team  D4C  won  both  the  Vegetation  Clearance  and  Surface  Clearance  events  and  was  awarded 
$500,000.  Team  UXOD  -  won  the  Geophysical  Mapping  event  and  the  overall  competition  for 
achieving  the  highest  total  score.  They  were  awarded  a  total  of  $1,250,000.  Team  Sky  Research 
did  not  qualify  for  any  awards  and  no  prizes  were  awarded  for  the  subsurface  clearance  event. 
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2.  INTRODUCTION 


This  report  documents  the  purpose,  background,  scope,  preparation,  and  execution  of  the 
Robotics  Range  Clearance  Competition  (R2C2)  sponsored  by  the  Joint  Ground  Robotics 
Enterprise  (JGRE),  the  US  Army  Corp  of  Engineers  (USACE),  and  the  Air  Force  Research 
Laboratory  (AFRL). 

2.1.  Purpose 

The  purpose  of  the  R2C2  was  to  quickly  tap  into  the  innovation  and  ingenuity  of  the  commercial 
robotic  technology  sector  to  improve  the  safety  and  effectiveness  of  any  or  all  of  the  four  tasks 
traditionally  associated  with  range  clearing  operations:  Vegetation  removal;  Surface  clearance; 
Geophysical  mapping;  and  Sub-surface  clearance.  The  ultimate  aim  was  to  be  able  to  clear  the 
millions  of  acres  currently  encumbered  with  spent  training  rounds  and  munitions  debris  and  to 
place  the  land  back  into  productive  use. 

2.2.  Background 

Range  clearance  operations  as  currently  conducted  are  manpower  intensive,  time  consuming, 
dangerous,  and  expensive.  Due  to  ever-restrictive  environmental  regulations,  the  DoD  has 
adopted  the  policy  of  building  new  ranges  on  existing  impact  areas.  In  addition,  existing  ranges 
require  target  maintenance,  line-of-sight  maintenance,  and  periodic  surface  clearance. 

Data  from  robotic  range  clearance  technology  development  efforts  indicated  the  strong  potential 
for  significant  reductions  in  the  time  and  cost  required  to  conduct  range  clearing  operations. 
Experiments  to  date  indicate  the  possibility  of  reducing  range  clearance  times  by  two  thirds  and 
costs  by  one  third  if  automated  clearing  equipment  is  used  (Skibba,  2003  -  Honey  Lake  Robotic 
Range  Clearance  Operations). 

Additionally  at  the  start  of  the  competition  process,  there  were  no  automated  “commercial  off  the 
shelf’  solutions  available  for  the  Department  of  Defense  to  procure.  The  traditional  approach  of 
establishing  a  research  and  development  (R&D)  program  that  could  be  transitioned  into  an 
acquisition  program  was  not  possible  within  the  desired  timeframe.  By  exercising  the  statutory 
authority  to  offer  a  cash  prize  for  research  and  development  achievements  for  this  robotic  range 
clearance  application,  it  seemed  possible  to  provide  the  desired  capability  in  a  significantly 
shorter  amount  of  time. 

(Section  2374a  of  title  10  United  States  Code  as  amended  by  Section  212  of  the  John  Warner 
National  Defense  Authorization  Act  for  Fiscal  Year  2007  Public  Law  109-364) 

Therefore  in  2009,  the  OUSD/AT&L  JGRE,  USACE,  and  AFRL  intiated  the  R2C2  to  be 
conducted  at  Camp  Guernsey,  Wyoming. 

2.3.  Scope 

The  tasks  associated  with  range  clearance  that  have  the  greatest  potential  for  applying  ground 
robotics  technology  include  automated  vegetation  clearance,  automated  surface  debris  clearance, 
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automated  geophysical  mapping,  and  automated  Sub-surface  anomaly  excavation.  The  purpose 
of  the  competition  was  to  assess  the  ability  of  the  competitor  systems  to  provide  increased  safety 
and  operational  effectiveness  to  range  clearance  operations.  The  competitor  systems  were 
expected  to  apply  robotics  technology  to  all  or  some  combination  of  the  inherent  tasks  in  a  range 
clearance  operation.  Because  the  competition  was  focused  on  increasing  safety  and  operational 
effectiveness  via  robotics  automation  as  well  as  reducing  time  and  cost,  competitors  were  not 
expected  to  attempt  to  develop  improved  vegetation  removal  tools  or  geophysical  detection  and 
identification  sensor  technology. 

2.4.  Goals  &  Objectives 

The  objective  of  this  competition  was  to  advance  robotic  technology  used  in  range  clearance 
operations  in  order  to  increase  operational  effectiveness  while  providing  greater  safety  for  range 
clearance  team  members.  The  technical  goal  was  to  advance  the  state  of  the  art  from  remote 
controlled  operation  up  to  fully  automated  robotic  operations.  The  Army  was  especially  looking 
for  solutions  and  approaches  to  the  range  clearance  tasks  that  would  yield  manpower,  cost, 
schedule,  and  safety  benefits  over  production  methods  currently  in  use.  While  full  automation 
was  desired,  it  was  not  required  to  actually  compete.  The  Anny  saw  the  competition  as  a  way  to 
prove  the  viability  of  the  methods  and  to  provide  data  that  would  assist  them  with  defining  their 
requirements  for  robotic  range  clearance. 
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3.  THE  COMPETITION 


The  JGRE  solicited  the  help  of  AFRL’s  Materials  and  Manufacturing  Directorate,  Airbase 
Technologies  Division’s,  Airbase  Engineering  Development  Branch  (AFRL/RXQE)  Robotic’s 
Group  at  Tyndall  Air  Force  Base  (AFB)  to  organize  and  manage  the  R2C2.  AFRL/RXQE  has 
extensive  experience  conducting  research  and  development  of  automated  range  clearance 
systems.  AFRL,  USACE,  and  JGRE  held  a  planning  meeting  in  September  2009  at  Camp 
Guernsey,  Wyoming.  Present  were  members  representing  AFRL,  JGRE,  USACE,  and  JTEC  that 
would  be  responsible  for  the  competition  planning,  logistics,  and  execution.  Camp  Guernsey  was 
chosen  as  the  competition  site  because  of  the  availability  of  terrain  suitable  for  vegetation 
clearance  and  the  ability  of  the  Camp  to  support  the  competition  events.  Figure  1  shows 
examples  of  Camp  Gurensey  terrain  that  were  inspected  during  this  initial  meeting. 


Figure  1.  Examples  of  Terrain  and  Vegetation  at  Camp  Guernsey 


The  planning  meeting  at  Camp  Guernsey  set  the  basic  structure  of  the  competition,  laid  out  a 
notional  timeline,  and  established  roles  and  responsibilities  for  conducting  the  competition.  The 
competition  director,  Mr.  Brian  Skibba  (AFRL/RXQE)  then  established  a  competition  oversight 
integrated  product  team  (IPT)  as  a  mechanism  to  oversee  competition  preparations  and  planning. 

A  request  for  information  (RFI)  to  industry  was  released  to  gauge  the  level  of  interest  of 
potential  competitors  and  solicit  opinions  from  industry  about  the  proposed  competition 
structure.  The  competition  was  publically  announced  with  a  kickoff  meeting  and  briefings  in 
Washington,  DC  in  October  2009,  followed  by  an  Industry  Day  at  Tyndall  AFB  in  December 
2009.  The  Kickoff  Meeting  and  the  Industry  Day  events  were  designed  to  provide  prospective 
teams  with  infonnation  about  the  proposed  schedule,  the  nature  of  the  tasks  involved,  the 
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competition  rules  as  then  envisioned,  lessons  learned  from  AFRL  about  robotic  range  clearance, 
and  to  answer  questions  about  the  nature  and  purpose  of  the  competition. 

The  remaining  sections  of  this  report  document  the  activities  and  events  leading  up  to  and 
including  the  competition  itself. 

3.1.  Competition  Kickoff  Meeting 

The  R2C2  Kickoff  Meeting  was  held  in  Washington,  D.C.  on  6  October  2009.  The  meeting  was 
publicly  announced  on  the  Fed  Biz  Ops  (www.fbo.gov)  website  inviting  anyone  with  an  interest 
in  the  competition  to  attend.  Seventy-five  people  attended  the  meeting  that  included  briefings 
introducing  the  competition,  the  expected  robotic  range  clearance,  indefinite  delivery-indefinite 
quantity  (IDIQ)  services  contract  with  the  US  Army,  describing  the  range  clearance  process, 
explaining  AFRL  lessons  learned  in  the  field,  and  describing  the  facilities  available  at  Camp 
Guernsey.  The  meeting  also  included  a  question  and  answer  (Q&A)  session  and  time  for  the 
attendees  to  network. 

Following  the  meeting,  an  informational  website  ( roboticrangeclearance.  com)  was  established 
(Figure  2)  to  publicize  and  inform  those  interested  in  the  competition.  A  list  of  the  attendees  of 
the  Kickoff  meeting  was  posted,  along  with  the  results  of  the  Q&A  session.  Eventually,  the 
website  would  serve  as  the  primary  communication  mechanism  with  the  public  as  all  versions  of 
the  rules  and  the  regularly  updated  Q&A  documents  would  be  posted. 


Robotic  Range  Clearance  Prize  Competition 


*1 


Vi* 


Introduction  to  the  Robotic  Range  Clearance  Competition 

Sponsored  by  *»•  Ground  Robobcs  CnUrpnw  coH«6or»»on  *n0i  li«  US  Army  Corps  of  Enfnoon  and  *>•  Air  Fore* 

”  comeeMign  mmmflicn  Announcements  (updated  as  of  30  August  201 1) 

»  QuesHons/Comments 

Ttie  Robotic  Range  Clearance  Competition  (R2C2)  Competitor  Winners  Announced 


The  Robotic  Range  Clearance  Competition  (R2C2)  Team,  the  Office  of  the  Secretary  of  Defense  (OSO).  Land  Warfare  and  Munitions 
the  Air  Force  Research  laboratory,  and  the  U  S  Army  Corps  of  Engineers  would  like  to  announce  the  winners  from  the  Robotic  Range 
Clearance  Competition  (R2C2)  held  August  7-14, 20 1 1  at  Camp  Guernsey.  WY 


The  Office  of  Secretary  of  Defense  (OSD)  m  collaboration  with  the  US  Air  Force  and  US  Army  conducted  a  Robotic  Range  Clearance 
Competition  (R2C2)  to  foster  the  ability  to  dear  training  ranges  of  debns  and  unexploded  ordnance  (UXO)  using  robotic  technologies 
The  purpose  of  the  Robotic  Range  Clearance  Competition  (R2C2)  was  to  quickly  tap  into  the  innovation  and  ingenuity  of  the 
commercial  robotic  technology  sedor  to  improve  the  safety  and  effectiveness  of  the  four  tasks  traditionally  associated  with  range 
deanng  operations  1 )  vegetation  Clearance.  2)  Surface  Clearance.  3)  Geophysical  Mapping,  4)  Subsurface  Clearance 

The  R2C2  Competition  teams  were 

1  T earn  UXOO  Automation  -  composed  of  Kairos  Autonomy  Autonomous  SokAons.  SAJC  Zonge  Engineenng.  vailon  WM 
Robots  VKR.  Inc.  and  John  Deere 

2  Team  D4C  -  composed  of  ECC.  QinebQ  North  Amenca  and  Bobcat 

3.  Team  Sky  Research 

The  results  of  the  R2C2  were  announced  17  August  at  the  Assoaation  of  Unmanned  vehide  Systems  International  (AUVSI)  North 
Amenca  201 1  conference  in  Washington  D  C  Team  D4C  won  both  the  Vegetabon  Clearance  and  Surface  Clearance  events  and  was 
awarded  $500  000  Team  UXOO  -  won  the  Geophysical  Mapping  event  and  received  and  additional  $1,000,000  for  achieving  the 
highest  overall  total  competition  score  Their  total  award  was  Si. 250.000 


The  R2C2  Competition  Dtredors  would  like  to  congratulate  the  competing  teams  and  thank  the  R2C2  judges  and  support  staff  for 
making  the  R2C2  a  successful  and  rewarding  event 


Contact 


Figure  2.  R2C2  Website 
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The  Kickoff  meeting  emphasized  that  the  competition  was  not  about  sensor  or  platform 
development,  but  it  was  an  integration  and  robotic  algorithm/behavior  challenge  (Appendix  K: 
2009  -  Robotics  Prize  Competition  Kick-off  Meeting  Briefing).  The  meeting  also  emphasized 
that  there  would  be  key  decision  points  along  the  way. 

The  Kick-off  Meeting  intended  to  make  a  determination  of  industry’s  interest  in  both  the 
competition  and  system  development  to  support  the  Army’s  need  for  robotic  range  clearance. 

The  high  level  of  attendance  at  this  meeting,  75  people  representing  more  than  50  companies 
attended,  and  responses  to  the  RFI,  1 8  total,  showed  that  there  was  sufficient  interest  to  continue 
with  the  competition.  Based  on  the  strong  interest  in  the  R2C2,  the  DoD  decided  to  proceed  with 
the  competition  and  offer  up  to  two  million  dollars  in  cash  prizes.  At  this  meeting,  it  was  also 
announced  that  the  systems  would  not  to  be  compared  against  each  other,  but  would  be  evaluated 
against  a  set  of  metrics  specified  for  each  of  the  designated  tasks.  The  competition  category  task 
objectives  would  be  finalized  and  announced  at  Industry  Day. 

3.2.  Industry  Day 

The  R2C2  Industry  Day  was  held  at  Tyndall  AFB  on  10  December  2009.  This  event  was  aimed 
at  providing  potential  competitors  with  lessons  learned  from  AFRL  robotic  range  clearance 
experience,  explaining  to  competitors  the  competition  rules  as  then  envisioned,  and  conducting 
limited  demonstrations  of  AFRL  range  clearance  technologies.  49  people  from  34  companies  and 
government  agencies  registered  for  the  Industry  Day. 

3.3.  Rules  and  Metrics  Formation 

The  R2C2  Oversight  IPT  created  a  Rules  and  Metrics  IPT  to  develop  the  details  of  competition’s 
rules  and  how  they  would  be  adjudicated.  A  two-day  meeting  was  held  in  Huntsville,  AL  with 
representatives  from  AFRL  and  their  support  contractors,  US  ACE,  and  the  JGRE  to  initiate  the 
competition  rules  development. 

The  R2C2  rules  were  driven  by  four  main  concerns:  safety  was  the  number  one  priority  (safety 
of  the  competitors  and  spectators  came  before  anything  else),  trying  to  appropriately  incentivize 
the  systems’  autonomy  level,  a  desire  to  not  structure  the  competition  to  favor  any  solutions  over 
others,  and  a  desire  to  keep  the  scoring  simple  so  the  teams  could  easily  understand  and  compute 
their  own  task  scores.  The  metrics  were  structured  around  assessing  range  clearance  performance 
outcomes,  level  of  autonomy,  and  man-hours  to  complete  the  tasks.  As  a  result,  while  range 
clearance  outcomes  were  emphasized,  autonomy  was  an  important  driver  in  the  evaluation. 

The  Rules  and  Metrics  IPT  produced  a  baseline  R2C2  Rules  and  Metrics  document  that  would 
be  periodically  reviewed  and  updated.  Appendix  A  includes  the  fifth  and  final  version  of  the 
Rules  and  Metrics  document  as  published  immediately  prior  to  the  competition. 

3.4.  Letter  of  Intent 

A  Letter  of  Intent  (LOI)  was  crafted  by  the  Oversight  IPT  and  the  JGRE.  The  purpose  of  this 
letter  was  to  obtain  commitments  to  compete  from  potential  teams.  After  legal  review  by  the 
OUSD,  the  LOI  was  finally  released  26  February  2010  to  the  competition  website  (Appendix  E: 
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Letter  of  Intent).  By  signing  the  letter,  each  team  and  its  members  agreed  to  have  a  mid-progress 
review  and  appear  at  the  R2C2  site  for  the  competition  at  a  mutually  agreed  upon  time,  ready  to 
perform.  It  was  not  intended  to  be  a  legally  binding  contract,  but  rather  an  expression  of  intent, 
requiring  written  notification  to  the  competition. 

The  R2C2  Oversight  IPT  received  12  LOIs  (Sky  Research,  Autonomous  Solutions  (2  members), 
Zonge  Engineering,  Millenworks,  SAIC,  Kairos  Autonomi  (4  members),  DOK-ing,  Redux, 
Battelle  (2  members),  ECC  (2  members),  Rogers,  and  Raytheon)  by  the  response  deadline  of  3 
May  2011.  These  totaled  18  original  team  members. 

3.5.  The  Teams 

During  the  time  between  the  LOIs  were  received  and  the  competition  event  some  of  the 
competitors  dropped  out  of  the  competition  for  various  reasons.  Some  companies  were  not  in  a 
position  to  self-fund  the  technology  development  necessary  for  them  to  compete.  Also,  several  of 
the  competitors  decided  to  collaborate,  which  resulted  in  many  individual  teams  becoming  a  few, 
large  teams.  As  of  the  spring  of  201 1,  only  three  teams  of  the  original  twelve  remained. 

Team  UXOD  Automation  (Figure  3)  was  comprised  of  Kairos  Autonomi,  Autonomous 
Solutions,  SAIC,  Zonge  Engineering,  Vallon,  WM  Robots,  VKR,  Inc.,  and  John  Deere  (Team 
UXOD  Automation  later  added  VKR,  Inc.  and  John  Deere  to  their  team). 

Team  D4C  (Figure  4)  was  comprised  of  ECC,  QinetiQ  North  America,  and  Bobcat. 

Team  Sky  Research  (Figure  5)  was  comprised  of  Sky  Research,  Inc. 

Both  Team  UXOD  Automation  and  Team  D4C  committed  to  compete  in  all  four  events  while 
Team  Sky  Research  only  planned  to  compete  in  the  Geophysical  Mapping  event,  which  made 
them  ineligible  for  the  Grand  Prize. 


Figure  3.  Members  of  Team  UXOD  Automation 
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Figure  4.  Members  of  Team  D4C 


Figure  5.  Members  of  Team  Sky  Research 


3.6.  Validation  Trials 

A  series  of  validation  trials  were  planned  by  the  R2C2  team  to  establish  a  baseline  level  of 
robotic  performance  possible  and  validate  the  competition  rules.  These  validation  trials  were  to 
be  conducted  at  Camp  Guernsey,  WY,  and  would  also  serve  to  provide  data  to  the  US  ACE 
regarding  robotic  range  clearance  system  performance  for  their  planned  robotic  range  clearance 
services  IDIQ  contract.  During  the  week  of  18-22  October  2010,  the  ARA  JTEC  group,  with  the 
University  of  Florida’s  (UF)  Center  for  Intelligent  Machines  and  Robots  (CIMAR)  lab, 
conducted  the  R2C2  validation  trials. 

The  objectives  of  the  validation  trials  were  to  validate  the  scoring  criteria  for  the  R2C2 
competition  and  establish  baselines  for  the  scoring  components  (e.g.,  expected  time  and 
manpower  requirements).  These  results  were  reported  to  the  R2C2  Oversight  IPT  to  serve  as  an 
empirical  basis  for  modifying  the  scoring  criteria  and  competition  procedures,  if  necessary.  The 
validation  trials  were  held  at  Camp  Guernsey’s  North  Training  Area,  located  north  of  Guernsey, 
WY,  shown  in  Figure  6. 
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Figure  6.  Map  of  Wyoming  Showing  General  Location  of  R2C2  Ranges 


ARA’s  JTEC  group  organized  the  logistics,  prepared  the  ranges,  integrated  the  automation 
hardware  with  the  All-purpose  Remote  Transport  System  (ARTS)  vehicles,  instrumented  the 
range  boundaries  with  video,  provided  the  system  tele-operator,  and  supplied  the  human- 
machine  interaction  referee  for  the  demonstration.  The  USACE’s  Andy  Schwartz  oversaw 
geophysical  mapping  tasks.  Mr.  Brian  Skibba  of  AFRL/RXQE  and  Mr.  Randy  Williams  of 
JGRE  represented  the  R2C2  Oversight  IPT  and  observed  the  trials.  ARA’s  Northeast  Division 
supported  the  baseline  and  demonstration  geophysical  mapping  by  providing  their  “Scout” 
geophysical  mapping  trailer  system  and  the  technical  expertise  to  operate  the  sensors  and  data 
collection  software.  The  University  of  Florida’s  CIMAR  lab  teamed  with  ARA  to  provide  the 
hardware  and  software  to  automate  the  range  clearance  machinery.  The  validation  trial 
participants  are  shown  in  Figure  7  with  an  autonomous  ARTS,  ARA’s  Scout  Trailer,  and  a 
robotic  Gyro-Trac  mulching  machine. 


Figure  7.  Attendees  of  the  R2C2  October  Validation  Trials 
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The  UF/ARA  team  acted  as  the  notional  competitor.  The  plan  was  to  demonstrate  some  level  of 
perfonnance  in  all  events  in  an  effort  to  exercise  the  competition  plan  and  to  test  the  scoring 
plan.  The  UF/ARA  team  had  a  mix  of  autonomy  levels.  For  geophysical  mapping,  they  used  a 
fully  autonomous  ARTS  towing  the  ARA  Scout  trailer  mentioned  earlier  (Figure  8). 


Figure  8.  ARTS  Towing  the  Scout  Trailer  During  the  Geophysical  Mapping  Event 


For  the  sub-surface  clearance  task,  a  tele-operated  ARTS  with  a  backhoe  attachment  was  used. 
The  system  employed  a  GPS  waypoint  system  to  drive  to  the  locations  where  the  industry 
standard  objects  (ISO)s  were  buried,  with  the  operator  using  tele-operation  to  accomplish  the 
digging  behavior  (Figure  9). 


For  the  surface  clearance  task,  a  semi-autonomous  ARTS  system  was  used  to  tow  a  Cherrington 
beach  cleaner  (Figure  10).  The  ARTS  again  used  GPS  waypoint  guidance  for  navigation  and 
steering  and  tele-operation  for  the  Cherrington  functions. 
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Figure  10.  ARTS  with  the  Cherrington  Beach  Cleaner  Attachment 

For  vegetation  clearance  task,  the  UF/ARA  team  used  a  fully  autonomous  GPS  waypoint  guided 
ARTS  pushing  a  mowing  deck  for  grass  and  small  vegetation  and  a  tele-operated  Gyro-Trac 
mulching  machine  for  trees.  The  outcomes  of  the  validation  trial  confirmed  that  the  facilities 
plan  and  the  design  of  the  competition  ranges  were  appropriate  for  the  competition  and  that  the 
basic  rules  and  metrics  were  appropriate  to  score  the  competition.  The  trials  did  point  to  some 
tailoring  of  the  scoring  criteria  and  other  competition  logistics  issues.  For  a  more  complete 
description  of  the  validation  trial,  (refer  to  Appendix  B:  Validation  Trials).  Some  of  the  report’s 
specific  recommendations  went  in  to  the  logistics  planning  for  the  actual  competition.  These 
included: 

•  The  competitors  will  be  given  the  opportunity  to  arrive  at  the  competition  1  week  prior  to 
the  start.  They  will  be  provided  practice  areas  to  fine  tune  and  debug  any  last  minute 
issues  or  failures  of  their  systems  during  this  time.  For  the  competition  they  will  then 
have  their  assigned  time  to  complete  the  competition  task.  The  only  exceptions  to  this 
will  be  for  weather,  safety,  or  military  priority  issues  that  might  delay  or  impede  the 
competition  activity. 

•  The  dig  map  provided  for  the  Sub-Surface  UXO  clearance  tasks  will  include  a  "target 
strength  value"  to  represent  a  realistic  geophysical  mapping  product.  The  makeup  of  this 
value  will  be  determined  through  consultation  with  Mr.  Andy  Schwartz,  USACOE 
Geophysicist.  In  addition  to  assigning  a  weighted  value  to  UXOs,  we  will  not  include  the 
“small”  type  ISOs  in  Sub-Surface  clearance  ranges.  However,  we  will  keep  the  total 
number  of  ISOs  at  50-60,  or  25-30  Large  ISOs  and  30  Medium  ISOs. 

•  A  meeting  between  the  JGRE  director  and  Camp  Guernsey  Garrison  Commander  is 
suggested  to  discuss  importance  of  this  activity  and  commitment  for  a  guaranteed  level  of 
lodging  and  support. 

•  The  Oversight  IPT  should  review  the  categories  of  human  interaction  to  ensure  that  they 
reflect  the  intent  of  the  competition.  Currently,  any  automated  behavior  at  all,  no  matter 
how  little,  results  in  25%  of  the  available  human  interaction  points. 

All  of  these  issues  were  implemented  or  addressed  for  the  actual  competition. 
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3.7.  In-Process  Reviews 


Each  of  the  competitor  teams  were  required  to  host  in-process  review  (IPR)  meetings.  These 
meetings,  held  at  the  competitor  team  facilities,  were  designed  to  provide  a  status  check  on  each 
team’s  safety  preparations  and  evaluate  the  likelihood  that  they  would  be  ready  for  the 
competition.  The  R2C2  Oversight  IPT  did  not  want  to  drive  teams  toward  particular  solutions,  so 
part  of  the  assessment  was  whether  the  teams’  proposed  solutions  were  going  to  fit  within  the 
rules,  but  not  to  judge  their  proposed  solutions.  At  all  IPR  meetings,  the  R2C2  Oversight  IPT 
members  stressed  that,  while  the  nature  of  the  tasks  were  range  clearance,  this  was  to  be  a 
robotics  competition,  which  emphasized  the  automation  of  these  tasks. 

3.7.1.  Teams  D4C  and  UXOD  Automation 

Team  D4C  and  Team  UXOD  Automation  demonstrated  a  good  understanding  of  the  functional 
requirements  and  the  equipment  needed  to  perform  tasks.  At  the  time  of  the  IPR  visits,  neither 
team  had  completed  plans  for  doing  subsurface  clearance  work  due  to  challenges  with 
reacquisition,  method  of  excavation  and  separating  seed  items  from  excavated  dirt.  Both  of  these 
teams  had  complete  geophysical  operation  plans  based  on  extensive  field  experience  and/or 
expert  guidance.  While  autonomous  control  was  readily  possible,  it  was  not  clear  at  that  point  to 
what  extent  their  equipment  would  be  operated  autonomously. 

3.7.2.  Sky  Research 

Team  Sky  Research’s  geophysical  mapping  plan  was  the  most  aggressive  plan  by  proposing  to 
build  and  integrate  a  waypoint  following  system  essentially  from  scratch.  Their  team  had 
established  credentials  in  the  geophysical  mapping  domain  and  appeared  confident  that  they 
could  succeed,  but  did  not  have  any  dedicated  robots,  engineers,  and  technicians  on  staff  at  that 
time. 


3.7.3.  General  Observations 

Team  UXOD  Automation  appeared  to  have  the  greatest  technical  experience  in  robotic  control, 
navigation  and  autonomous  operations,  but  they  were  less  experienced  in  actual  field  work. 
Team  D4C  appeared  to  have  a  better  combination  of  technical  and  actual  UXO  field  experience 
to  successfully  accomplish  the  competition  tasks.  Team  Sky  Research  appeared  to  have  an 
expertise  advantage  in  the  geophysical  mapping  task. 

3.8.  Logistics  Effort 

Hosting  the  competition  required  a  large  amount  of  logistical  effort  and  equipment  to  facilitate 
the  teams,  judges,  guests,  and  execution  of  the  competition.  The  main  base  of  control  was  a 
paddock  set  up  to  the  northeast  of  the  range  area. 

3.8.1.  Paddock 

The  paddock  was  a  fenced  area  that  was  the  main  control  base  for  the  competition  (Figure  1 1). 
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PARKING  AREA 


Figure  11.  Diagram  of  the  Paddock  Layout 


The  teams  and  competition  personnel  worked  in  the  paddock  area  when  not  on  the  ranges.  Two 
hard  sided  work  trailers  were  provided  for  command  and  control  and  judge’s  work  space.  The 
trailers  had  electricity,  climate  control,  radios  for  communicating  with  personnel  in  the  area,  and 
served  as  the  entry  control  point  for  teams  to  check  in  and  coordinate  movements  with  command 
and  control.  The  paddock  also  included  a  mess  tent  where  food  was  served.  Three  meals  were 
served  every  day,  with  most  days  having  the  option  of  a  hot  lunch  or  sack  lunch.  Each  team  had 
a  pit  area  where  they  stored  and  worked  on  their  vehicles.  Teams  were  provided  with  two  tents  in 
which  to  work,  and  power  supplied  from  the  main  generator. 

A  large  restroom  trailer  was  rented  for  the  paddock  to  provide  adequate  restroom  facilities.  The 
paddock  also  contained  light  towers  for  night  work,  fuel  tanks  for  refueling  competition  vehicles, 
and  gravel  roads  and  paths.  The  compound  was  fenced  and  had  a  security  guard  each  night  to 
protect  the  competitor’s  equipment.  Figure  12  shows  the  actual  paddock  area  during  the 
competition. 


Figure  12.  Paddock  in  Use  during  the  Competition 
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3.9.  Range  Logistics 


In  addition  to  setting  up  and  seeding  the  ranges  with  simulated  UXO  items  as  required  by  the 
rules,  several  logistical  concerns  were  also  addressed  in  preparation  for  the  competition.  Each 
range  had  a  tent  to  work  in,  and  a  work  trailer  was  placed  at  a  central  location  for  guests  to  view 
the  operations.  All  personnel  and  equipment  movement  to  and  from  the  ranges  was  carefully 
controlled  by  command  &  control  and  each  range  boss. 

3.10.  Task  List 

The  following  is  the  high-level  task  list  from  the  R2C2  integrated  master  schedule.  The  list 
illustrates  the  comprehensiveness  of  the  preparation  for  the  competition. 

•  R2C2  Competition  Integrated  Master  Schedule 

•  Assign  staff  to  roles 

•  Competition  staff  uniforms 

•  Complete  Environmental  Assessment  or  Rec  &  Check 

•  Team  billeting/  Lodging 

•  Competitor  IPRs 

•  Pass  radios  through  spectrum  management 

•  Improve  road  network 

•  Create  Competition  Village 

•  Hard-sided  worksite  trailer  for  R2C2  mgmt/support 

•  Hard-sided  worksite  trailer  for  JGRE/Chief  Judge/Distinguished  Visitors 

•  Mess  tent 

•  Maintenance  tent 

•  Command  &  Control  trailer  (JTEC  Balboa) 

•  Communications 

■  Arrange  for  high-speed  internet  connections 

■  Setup  and  test  JTEC  radio  network 

■  Setup  Public  Address  System 

•  Mens/womens  latrines 

■  Setup  port-a-lets  at  each  range 

■  Setup  main  men/women  latrine  trailer  by  worksite  trailers 

■  Arrange  for  latrine  servicing 

•  Create  Paddock 

■  Survey  paddock  area 

■  Install  fencing 

■  Position  generators 

■  Lighting  trailers 

■  Fuel  Dump 

■  Establish  fueling  area 

•  Create  fueling  procedures 

•  Plan  daily  fuel  needs 

•  Create  Competition  Range  Area 

•  Signage 
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•  Arrange  daily  transportation  to  site 

•  Meals 

•  Emergency  Medical  Services 

•  Construct  Competition  Ranges 

■  Survey  and  mark  range  boundaries 

■  Prepare  ISOs  and  seed  items 

■  Seed  ranges  according  to  seeding  and  Quality  Control  (QC)  plan 

•  Judge  training 

•  Unloading 

•  Security 

•  Final  Competitor  Information  Packets 

•  Compile  individual  competitor  information  packets 

•  Deliver  to  teams 

•  Competitor  Practice 

•  Conduct  Live  Competition 

•  Pre-competition  briefing 

•  Competitor  safety  checks 

•  Team  D4C  Setup 

•  Team  D4C  Geophysical  Mapping 

•  Team  D4C  Vegetation  Clearance 

•  Team  D4C  Surface  Clearance 

•  Team  D4C  Sub-Surface  Clearance 

•  Team  UXOD  Setup 

•  Team  UXOD  Geophysical  Mapping 

•  Team  UXOD  Vegetation  Clearance 

•  Team  UXOD  Surface  Clearance 

•  Team  UXOD  Sub-Surface  Clearance 

•  Team  Sky  Research  Setup 

•  Team  Sky  Research  Geophysical  Mapping 

•  Deliver  meals  according  to  daily  schedule 

•  Media  and  Distinguished  Visitor  Day 

3.11.  Required  Equipment/  Supplies 

The  following  list  shows  the  equipment  items  that  were  needed  for  purchase  or  rental  to  illustrate 
what  was  needed  to  conduct  the  competition. 

•  Meals,  kickoff  dinner,  snacks/drinks 

•  Water 

•  Signs 

•  Paddock  Trailers 

•  Other  paddock  equipment:  toilets,  dumpster,  fuel  tanks,  generators,  lights,  tents,  fencing, 
con-ex,  cords,  spill  kits,  garbage  cans,  fire  kits,  rope,  steel  posts,  gravel 
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•  Command  &  control  needs:  Wifi  Aircards,  name  tags,  clocks,  lightning  detectors, 
furniture,  Public  Announcement  (PA)  system,  Global  Positioning  System  (GPS)  supplies, 
Radios 

•  Services  needed:  Ambulance,  security,  electrician,  fire  truck 

•  Personnel  clothing 

•  Tractor  w/backhoe,  mower,  digger 

•  Rental  Cars-  Pickups,  SUVs,  Vans,  and  Fuel 

•  Tele-handler 

•  Motels,  billeting 

•  Golf  Carts 

•  Boundary  cameras,  trailers 

•  Sensor  components 

•  Recreational  Vehicles/Trailers  for  on-site  lodging 

3.11.1.  The  Competition  Schedule 

Once  the  number  of  competitors  was  finalized,  the  extent  and  time  required  for  the  competition 
was  detennined  be  two  weeks  including  5-7  practice  days  the  week  prior  to  the  competition.  The 
early  August  time  frame  was  chosen  because  of  the  constraints  at  Camp  Guernsey.  Camp 
Guernsey  is  an  Army  National  Guard  staffed  Joint  Training  facility  and  as  such,  on-Camp 
billeting  is  typically  in  short  supply  from  May  through  the  end  of  July.  Table  1  shows  the 
competition  schedule  that  was  originally  proposed. 


Tal 

ble  1.  Original  R2C2  Schedule 

Activity 

Date 

Time 

Teams  arrive  at  Camp 
Guernsey 

3 1  July  -  7  August 

2011 

varies 

Practice  site  open 

1-5  August  2011 

0800-1700 

Judges  arrive  at  Camp 
Guernsey 

3  August  2011 

Varies 

Judges  training  at  site 

4-6  August  2011 

0900-1600 

Safety  Testing 

6-7  August  2011 

0900-1600 

Opening 

Reception/Dinner 

7  August  2011 

1800-2000 

Competition  Events 

8-10  August  2011 

0600-1700 

Media  &  Visitor  Day 

1 1  August  20 1 1 

0900-1400 

Competition  Events 

12-13  August  2011 

0600-1700 

Final  scoring  and 
adjudication 

14-15  August  2011 

0800-1800 

Personnel  depart 

16  August  2011 

varies 

Present  Results  at  the 
Association  for 
Unmanned  Vehicle 
Systems  International 
(AUVSI)  conference 

17  August  2011 

1600-1700 
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The  competition  schedule  began  with  a  practice  week  (planned  for  Monday,  1  August  through 
Friday,  5  August)  followed  by  a  day  off  on  Saturday,  6  August.  The  competition  was  kicked  off 
with  a  meeting  and  barbeque  on  Sunday,  7  August  and  the  events  running  from  Monday,  8 
August  through  Saturday,  13  August.  There  were  two  overrun  days  built  in  to  the  schedule,  the 
14th  and  15th  of  August,  in  case  of  inclement  weather. 

The  only  change  to  the  schedule  was  the  extension  of  practice  week  from  the  fifth  of  August 
through  the  afternoon  of  the  seventh.  The  R2C2  competition  director  elected  to  extend  practice 
week  due  to  inclement  weather  delays  during  the  scheduled  practice  week.  In  addition,  the 
competitor  teams  all  underestimated  their  need  to  practice  and  before  they  arrived  on  sight.  As  a 
result,  all  the  teams  were  significantly  behind  schedule  and  unprepared  for  the  competition  at  the 
end  of  the  regularly  scheduled  practice  week.  The  R2C2  had  a  two-day  contingency  built  into  the 
schedule  (through  the  19th)  in  case  of  more  serious  weather  delays.  Fortunately,  the  weather 
during  competition  week  was  good  and  the  contingency  weather  days  weren’t  needed. 

The  notional  competitor  schedule  is  shown  in  Figure  13. 


Day/Team 

Team  1 

Team  2 

Team  3 

Aug  7 

SUN 

Safety  Test 

Safety  Test 

Safety  Test 

Aug  8 

MON 

MAP 

VEG 

Aug  9 

TUE 

MAP 

Aug  10 

WED 

VEG 

MAP 

Aug  11 

THU 

Media  and  Visitor  Day 

Aug  12 

FRI 

SURF 

SUB 

Aug  13 

SAT 

SUB 

SURF 

Aug  14 
Aug  15 

SUN 

MON 

Weather  Days 

Figure  13.  R2C2  Competition  Week  Schedule 


Team  1  was  Sky  Research,  Teams  2  and  3  were  decided  by  a  blind  hat  pull  during  the  practice 
week.  Team  2  was  Team  D4C  and  Team  3  was  Team  UXOD  Automation.  The  daily  event 
schedule  is  shown  in  Table  2. 


Table  2.  Daily  Event  Schedule 


Activity 

Time 

Safety  Brief 

0545-600 

Teams  allowed  on  event  site  for  range  walk 
and  setup/breakfast 

0600-0800 

Competition  event  begins 

0800 

Optional  lunch  break 

varies 

Competition  event  ends 

1600-1700 

Initial  scoring  and  data  validation, 
Documentary  video  editing 

Endex-2100 
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Each  competition  event  lasted  8  working  hours  from  the  event  start  with  up  to  1  hour  of  optional 
sustenance  breaks  for  a  total  of  9  hours  maximum.  If  team  personnel  worked  on  any  equipment 
or  computers  during  the  sustenance  break,  that  time  was  charged  as  working  time. 

3.11.1.1.  The  Competition  Judges 

The  judging  duties  were  distributed  across  six  official  judges  for  the  competition. 

They  were: 

•  Chief  Judge  -  Dr  James  Overholt  -  United  States  Army/  Tank  and  Automotive  Research 
Development  and  Engineering  Center  (USA/TARDEC), 

•  Safety  Judge  -  Mr.  Lucas  Martinez  -  United  States  Air  Force/  Air  Force  Research 
Laboratory  (USAF/AFRL), 

•  Autonomy  Judge  -  Mr.  Michael  Bruch  -  United  States  Navy/  Space  and  Naval  Warfare 
Systems  Center  -  Pacific  (USN/SPAWAR), 

•  Geophysical  mapping  Judge  -  Mr.  Andrew  Schwartz  -  United  States  Anny  Corps  of 
Engineers  (USA/USACOE), 

•  Task  Judges  -  Mr.  Walter  Waltz  USAF/AFRL  and  Mr.  Plyer  McManus  USA/USACOE. 

The  duty  of  each  judge  was  to  oversee  the  events  for  their  responsible  areas  and  then  to 
adjudicate  the  scoring  for  that  event.  Data  collection  assistants  and  field  referees  assisted  the 
judges. 

•  The  Chief  Judge  was  responsible  to  supervise  the  judges,  data  collection  assistants,  and 
field  referees,  verify  and  validate  the  scoring  of  all  events,  and  interpret  the  rules  as 
applied  to  competition  adjudication  matters. 

•  The  Safety  Judge  was  responsible  to  conduct  and  supervise  all  of  the  safety  tests,  verify 
and  validate  the  safety  test  methods,  and  observe  all  of  the  team  events  to  judge  any 
perceived  safety  violations  or  issues. 

•  The  Autonomy  Judge  was  responsible  to  supervise  the  autonomy  data  collection 
assistants  and  observe  all  of  the  team  events  and  related  autonomy  data  to  judge  the  level 
of  autonomy  of  the  robotic  systems. 

•  The  Geophysical  mapping  Judge  was  responsible  to  supervise  the  geophysical  mapping 
data  collection  assistants,  verify  the  condition  of  the  event  site,  observe  all  of  the 
geophysical  mapping  events,  collect  the  geophysical  mapping  data  from  the  teams,  and 
process/calculate  the  geophysical  mapping  data  to  judge  the  performance  to  the  rules. 

•  The  Task  Judges  were  responsible  to  supervise  the  data  collection  assistants,  verify  the 
condition  of  the  event  sites,  observe  all  of  the  team  events,  inspect  the  condition  of  the 
sites  and  debris/collection  areas  post  event,  and  to  judge  the  task  perfonnance  for  each 
event. 

3.11.1.2.  R2C2  Team 

The  R2C2  team  (Figure  14)  included  the  Director  of  the  JGRE,  Mr.  Rob  Maline;  his  support 
from  Booz  Allen  Hamilton;  the  competition  judges;  the  competition  Director;  the  safety 
assessment  team  made  up  of  members  of  Wintec  Inc.;  the  video  production  team  made  up  of 
members  of  Integration  Innovation,  Inc.  and  the  ARA/JTEC  group. 
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Figure  14.  The  R2C2  Team 


3.11.2.  Preparing  the  Ranges 

An  overhead  layout  of  the  practice  and  competition  ranges  can  be  seen  in  Figure  15.  Individual 
practice  ranges  were  constructed  for  each  competition  task  and  designated  to  each  team  (denoted 
by  the  prefix  Pi,  2,  and  3).  For  the  actual  competition,  each  team  was  assigned  their  own 
vegetation  clearance,  surface  debris,  and  sub-surface  clearance  range  (denoted  by  the  prefix  R1 
&  2).  All  teams  used  the  same  geophysical  mapping  competition  range  for  the  geophysical 
mapping  task. 


R1SUB 


P1SUB 


R2SUB 


P2SUB 


RGEO 


P2GEO 


P1SUR 


P2SUR 


R1SUR 


R2SUR 


Safety 


Ran([e 


P1GEO 


Paddock 


PIVECj 


: 


H I V I G 


Legend 

I  I  (Pl,P2,P3)GEO-  Geophysical  Mapping  Practice  Range 
I  I  (P1,P2)VEG  -  Vegetation  Clearance  Practice  Range 
I  I  (P1,P2)SUR  -  Surface  Clearance  Practice  Range 
I  I  (P1,P2)SUB  -  Sub-surface  Clearance  Practice  Range 
□  Paddock 

I  I  C&C  -  Command  and  Control 


I  I  RGEO -Geophysical  Mapping  Competition  Range 
f  (R1,R2)VEG- Vegetation  Clearance  Competition  Range 
I  I  (R1,R2)SUR- Surface  Debris  Clearance  Competition  Range 
I  I  (R1,R2)SUB- Sub-Surface  Clearance  Competition  Range 
Q  Safety  Test  Range 
i  I  Fence  Buffer 


Figure  15.  R2C2  Competition  Map  Overview 
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The  ranges  were  allocated  as  follows: 

•  PI  (GEO,  SUB,  SUR,VEG)  &  R1(SUB,  SUR,  VEG)  -  Team  UXOD  Automation 

•  P2(GEO,  SUB, SUR,  VEG)  &  R2(SUB,  SUR,  VEG)  -  Team  D4C 

•  P3  GEO  -  Team  Sky  Research 

•  RGEO  -  All  Teams 

3.11.2.1.  Vegetation  Clearance  Ranges 

The  vegetation  clearance  ranges  were  collocated  near  each  other  to  best  represent  one  another  in 
terrain  and  vegetation  variation.  Although  the  vegetation  ranges  were  not  identical,  they  were 
constructed  in  a  manner  to  ensure  no  competitor  had  an  advantage  (Figure  16  shows  these 
ranges). 

Each  range  contained  an  equal  amount  of  grass/brush  for  mowing  and  the  same  number  of  trees 
to  be  removed.  The  vegetation  clearance  ranges  comprised  of  a  3-acre  plot.  The  first  half  was 
relatively  flat  and  predominantly  covered  with  grass,  while  the  second  half  lay  on  a  slope  with 
ravines  and  variations  of  vegetation  (grass,  shrub,  and  trees). 


Team  D4C  Tree  Removal  Area  Team  UXOD  Automation  Tree  Removal  Area 

Figure  16.  Mowing  and  Tree  Removal  Areas  of  Vegetation  Clearance  Ranges 


Team  D4C  Mowing  Area 


Team  UXOD  Automation  Mowing  Area 


3.11.2.2.  Geophysical  Mapping  Range 

All  teams  shared  the  same  geophysical  mapping  range  (Figure  17)  during  the  competition.  The 
geophysical  mapping  range  was  constructed  over  an  area  of  1.6  acres.  Two  obstacles  were  placed 
within  the  area  reducing  the  required  coverage  area  to  approximately  1.58  acres.  One  hundred 
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and  twenty  six  ISOs  (63  small,  58  medium,  and  5  large)  were  buried  in  known  locations 
throughout  the  range  according  to  the  R2C2  Geophysical  Mapping  Range  Seeding  Work  Plan 
and  QC  Plan  document.  All  pipe  sections  were  buried  in  a  vertical  orientation,  which  produced 
single  mono-pole  anomaly  in  horizontal  loop  electromagnetic  induction  metal  detectors.  These 
seed  items  can  be  seen  in  the  right  side  of  Figure  17.  Later  during  the  competition,  the 
geophysical  mapping  range  was  reduced  by  approximately  25%  to  better  meet  the  R2C2’s 
objectives  of  having  all  competitors  complete  a  designated  area  within  the  allotted  time. 


Figure  17.  Common  Geophysical  Mapping  Range  and  ISO  Seed  Items 


3.11.2.3.  Surface  Clearance  Ranges 

The  surface  clearance  ranges  were  located  adjacent  to  each  other  to  minimize  variations  in 
terrain  to  the  extent  possible.  The  surface  clearance  ranges  were  both  comprised  of  five  acres 
with  two  railroad  tie  obstacles  each  (Figure  18  shows  what  these  ranges  looked  like.) 

Each  range  was  mowed  in  preparation  for  seeding  and  was  seeded  with  120  small  ISOs,  120 
medium  ISOs,  and  310  standard  pieces  of  steel  plate  and  steel  angle. 


Figure  18.  Surface  Clearance  Ranges:  Team  D4C  Range  (left); 
Team  UXOD  Automation  Range  (right) 


The  seeding  was  randomly  distributed  to  simulate  a  real  impact  range  with  UXO  and  target 
debris.  The  seed  items  can  be  seen  in  Figure  19. 
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Figure  19.  Surface  Clearance  Range  Seed  Items  (ISOs  and  Steel  Plate  and  Angle) 


3.11.2.4.  Sub-Surface  Clearance  Ranges 

The  sub-surface  clearance  ranges  were  also  located  adjacent  to  each  other  to  minimize  variations 
in  terrain  to  the  extent  possible.  The  sub-surface  clearance  ranges  were  both  two  acres  (Figure 
20). 

Each  range  was  mowed  and  then  seeded  with  25  large  ISOs  and  25  medium  ISOs  in  a  grid 
pattern.  The  ISOs  were  buried  in  the  same  manner  as  the  ISOs  for  the  Geophysical  Mapping 
Range.  All  of  the  ISO  locations,  depths,  and  orientations  were  recorded.  Two  weeks  prior  to  the 
start  of  the  competition,  the  teams  were  provided  the  seed  item  locations  in  Universal  Transverse 
Mercator  (UTM)  coordinates..  This  information  included  “shapefiles”  of  their  ranges,  a  dig  list 
for  their  sub-surface  range,  and  descriptions  for  nomenclature  and  file  naming  conventions.  The 
dig  list  information  included  approximate  ISO  size  information,  approximate  depth  range,  local 
UTM  dig  coordinates,  and  horizontal  error  infonnation  (30  cm). 


Figure  20.  Sub-Surface  Clearance  Ranges:  Team  D4C  Range  (left); 
Team  UXOD  Automation  Range  (right) 


3.12.  Safety  Test  Requirements 

Before  each  team  could  begin  any  operations  on  the  practice  ranges  and  participate  in  the 
competition,  they  were  required  to  demonstrate  their  robotic  systems  met  all  of  the  R2C2  safety 
requirements..  The  following  safety  requirements  for  each  of  the  competitor’s  robotics  system 
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were  observed  and  evaluated  with  a  pass  or  fail  according  to  the  “R2C2  Competition  Rules  and 
Metrics  Document”  found  in  Section  3  of  the  R2C2  Rules  and  Metrics  (Refer  to  Appendix  A). 

Emergency  Stop:  The  system  shall  have  an  Emergency  Stop  (E-Stop).  The  system  must  halt 
within  15  meters  and  cease  all  equipment  operations  when  the  E-Stop  is  initiated. 

Warning  Devices:  Each  vehicle  shall  be  equipped  with  a  warning  light  that  is  activated 
according  to  the  state  of  the  E-Stop  system.  Each  vehicle  shall  display  one  or  more  flashing 
amber  warning  lights,  the  combination  of  which  results  in  visibility  360  degrees  azimuthally 
around  the  vehicle.  The  warning  light(s)  shall  operate  when  the  vehicle  is  not  in  an  E-stop  state, 
i.e.,  RUN  mode.  The  vehicle  may  not  commence  movement  until  the  warning  light(s)  has  been 
in  operation  for  5  seconds.  The  warning  light(s)  shall  comply  with  SAE  Class  1  standards  for 
warning  lights  and  shall  not  produce  light(s)  than  can  be  confused  with  those  of  public  safety 
vehicles  such  as  law  enforcement,  fire,  or  ambulance. 

Loss  of  Communications  Stop:  The  system  shall  automatically  halt  and  cease  operations  if 
communications  with  the  system  are  lost  or  interrupted  for  a  maximum  of  2  seconds  and  may 
travel  no  farther  than  30  meters. 

No  Freewheel:  The  systems  shall  not  be  capable  of  motion  when  stopped  or  un-powered.  For 
example,  systems  that  would  roll  downhill  if  shut  off  are  considered  freewheeling  and  are  un¬ 
safe  for  competition. 

If  a  test  was  failed,  the  team  was  allowed  to  address  the  problem  and  retest  the  system  until  it 
passed  given  that  there  was  sufficient  time  left  in  the  competition  schedule,  and  that  the  R2C2 
director  was  satisfied  that  the  system  perfonnance  was  repeatable. 

If  a  competitor  had  made  any  significant  hardware  or  software  changes  anytime  during  the 
competition  they  were  required  to  re-qualify  that  particular  robotic  system  to  meet  all  R2C2 
safety  requirements.  Hardware  and  software  changes  were  defined  as  the  following: 

•  Hardware  changes  -  Any  hardware  configuration  changes  to  the  robotic  system  that 
could  render  it  unsafe  during  static  and  mobile  operation;  changing  tool  implements  (like 
tree-cutting  shears,  buckets,  mulching  heads,  etc.)  were  not  considered  a  qualifying 
change  as  long  as  they  were  an  OEM  or  aftennarket  part  designed  for  the  system. 
Electrical  hardware  changes  such  as  electronic  sensor  and  control  modules  that  were  part 
of  the  robotic  control  system  were  considered  to  be  significant  changes  and  would  require 
a  retest. 

•  Software  changes  -  Any  software  source  code  change  that  could  compromise  the  safety 
of  the  robotic  system  according  the  R2C2  safety  metric.  The  teams  were  required  to 
report  any  software  changes  for  the  R2C2  management  to  evaluate  and  detennine  if  a 
retest  was  required. 

A  sample  of  the  safety  evaluation  forms  are  found  in  Appendix  G:  Safety  Evaluation  Form. 
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3.12.1.  Safety  Test  Setup 

All  of  the  safety  requirements  except  the  freewheel  test  were  conducted  at  a  distance  of  800 
meters  away  from  the  operator  control  unit  (OCU).  The  tests  (e-stop,  loss  of  communications 
stop,  and  warning  devices)  were  conducted  at  this  distance  because  each  robotic  system  was 
required  to  be  operable  from  800  meters.  This  requirement  was  intended  to  replicate  the  work 
constraints  of  a  real-world  robotic  range  clearance  operation,  where  personnel  must  be  located  at 
a  remote  area  to  provide  a  safe  standoff  from  the  blast  fragmentation  radii  of  UXO. 

3.12.1.1.  Safety  Base  Equipment  Setup 

Figure  21  illustrates  how  the  safety  range  and  equipment  was  setup  to  validate  each  competitor’s 
robotic  system.  An  enclosed  trailer  was  used  to  house  the  competitor’s  OCU,  the  safety  test 
equipment,  radio  frequency  (RF)  communication  system,  120VAC  generator,  and  provide  shelter 
from  adverse  weather  conditions. 


Figure  21.  Safety  Test  Setup 


The  test  equipment  was  built  into  a  pelican  case  enclosure  containing  a  fiber  optic  media 
converter,  customized  electronics  and  firmware  entitled  “RX”.  This  system  controlled  the  power 
provided  to  the  competitor’s  RF  communications  link  used  for  the  “loss  of  communications 
stop”  test.  The  RX  box  was  used  to  acquire  the  laser  data  measurements  and  display  real-time 
information  onto  two  monitors  for  the  tests  conducted  downrange.  The  monitors  were  setup 
inside  the  trailer  to  allow  the  safety  officer  to  easily  observe  and  evaluate  the  tests.  Additionally, 
each  test  was  recorded  onto  a  digital  video  recorder  (dvr.)  fde  to  review  any  disputes  protested 
by  the  competitor,  and  for  record  and  review  by  the  safety  officer.  Figure  22  shows  the  trailer 
where  the  tests  were  observed. 
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3.12.2.  Downrange  Test  Equipment 

The  safety  test  equipment  located  downrange  comprised  of  a  120V AC  generator,  laser  altimeter, 
a  second  pelican  case  entitled  “TX”  that  housed  a  fiber  optic  media  converter  with  RS-232  data 
and  video  input  for  the  laser  data  and  test  video,  respectively.  The  data  was  hardwired  via  fiber 
optic  cable,  which  was  transmitted  to  the  RX  box  located  at  the  base  station.  A  laser  altimeter, 
being  used  as  a  rangefinder,  was  used  to  accurately  measure  the  distances  travelled  for  the  e-stop 
and  loss  of  communications  stop  test.  The  laser  altimeter  was  mounted  on  a  tripod  and  bore- 
sighted  prior  to  the  competition.  Figure  23  shows  the  test  equipment  setup  in  the  field  and  the 
laser  altimeter  on  the  tripod. 


Figure  23.  Downrange  Test:  Test  Equipment  Setup  (left);  Laser  Ranger  Finder  (right) 


3.12.3.  No  Freewheel  Test 

The  no  freewheel  test  site  was  located  on  a  hillside  next  to  the  safety  test  base  station.  The 
hillside  worked  well  as  it  naturally  met  the  10-12  degree  slope  for  the  test  requirement  (Figure 
24). 
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Figure  24.  No  Freewheel  Test:  Diagram  (left);  Vehicle  on  Freewheel  Test  Hell  (right) 


3.13.  Team  Arrival  and  Practice  Week 

Teams  began  arriving  at  Camp  Guernsey  on  1  August  2011.  Upon  arrival  to  Camp  Guernsey 
each  team  checked  in  with  the  JTEC  office  to  pick  up  their  final  competition  information  packets 
and  their  room  keys.  All  of  the  competitor  team  members  were  provided  lodging  at  Camp 
Guernsey.  The  packets  included  R2C2  staff  contact  information,  medical  and  safety  contact 
information,  competition  schedule,  directions  to  the  competition  site,  important  Camp  Guernsey 
regulations  and  procedures,  and  general  infonnation  about  area  eateries,  stores,  gas  stations, 
pharmacies  and  hospitals.  The  packet  is  included  in  Appendix  B. 

After  check  in,  each  team  transported  their  equipment  to  the  competition  paddock  area,  unloaded 
their  equipment  and  positioned  it  in  their  designated  pit  areas.  The  competition  director  and  the 
safety  judge  conducted  walk  around  inspections  to  access  the  overall  safety  features  of  the 
robotic  systems  such  as  warning  lights,  e-stops,  etc. 

Before  each  R2C2  competitor  could  begin  operating  on  the  practice  ranges  and  participate  in  the 
competition,  they  were  required  to  demonstrate  their  robotic  systems  could  meet  all  of  the  R2C2 
safety  requirements  at  the  safety  test  range.  If  a  vehicle  failed  a  test  the  competitor  was  allowed 
to  modify  their  systems  and  rerun  the  tests  as  often  as  needed,  but  would  have  to  pass  all  safety 
requirements  before  being  allowed  to  practice  and  compete. 

The  competitors'  robotics  systems  were  tested  and  observed  for  the  following  safety 
requirements  and  evaluated  with  a  pass  or  fail  according  to  the  Rules  section  of  the  “R2C2 
Competition  Rules  and  Metrics  Document"  found  in  Appendix  A. 
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4.  COMPETITION  RESULTS  AND  DISCUSSION 

4.1.  Safety  Test  Results 

The  safety  tests  were  completed  successfully  for  all  robotic  vehicles,  prior  to  competition.  Most 
of  the  system’s  testing  was  completed  prior  to  the  practice  events  and  since  no  significant 
changes  were  made  most  systems  did  not  have  to  be  re-qualified.  Team  Sky  Research’s  system 
failed  the  initial  no  free  wheel  test  and  had  to  have  the  brake  actuation  system  modified.  This 
system  passed  this  test  successfully  with  the  modification.  Team  UXOD  Automation  made  a 
significant  hardware  change  to  their  feller  buncher  system  by  adding  a  large  external  generator  to 
it  between  the  surface  clearance  and  sub-surface  clearance  events.  Due  to  this  change,  they  were 
required  to  complete  all  safety  tests  for  this  system  again.  This  was  accomplished  the  day  prior  to 
their  sub-surface  clearance  event  and  the  system  successfully  passed  all  the  tests  in  the  new 
configuration. 

4.2.  Competition  Site  Access 

The  teams  were  not  allowed  access  to  the  event  ranges  until  the  morning  of  the  event.  Starting  at 
0600  the  competing  team  was  allowed  to  begin  setting  up  their  equipment  on  the  event  range  and 
conduct  a  site  walk.  They  were  allowed  to  survey  the  terrain,  and  measure  items  of  interest  such 
as  trees,  ravines,  boundaries,  etc.  however,  they  were  not  allowed  to  physically  mark  anything 
within  the  boundary.  The  competitors  also  used  this  time  to  set  up  their  operational  area  and 
ready  their  machines  to  begin  the  competition  at  0800. 

Competition  time  was  planned  for  0800-1700  and  would  be  offset  pending  any  weather 
interruptions.  Each  team  was  allotted  8  hours  of  operational  time  and  along  with  an  optional  one- 
hour  break  (for  lunch,  rest,  etc.)  if  the  competitor  deemed  necessary  however  during  break  times, 
no  maintenance  or  robotic  activity  could  be  performed. 

In  order  for  a  team  member  to  go  down-range,  they  would  have  to  E-Stop  all  robotic  systems  and 
request  permission  from  the  competition  director.  This  simulates  the  real-world  operation  of 
requesting  Explosive  Ordnance  Disposal  (EOD)  escort  support  prior  to  entering  any  UXO  range 
area.  Any  time  used  during  a  down  range  event  counted  against  the  overall  8-hour  runtime 
allotted. 

In  the  event  an  E-Stop  was  initiated  by  any  of  competition  staff  (judges,  range  boss,  director, 
etc),  time  was  stopped  for  the  competitor  until  the  situation  was  assessed  and  operations  were 
authorized  to  resume  either  by  the  Range  Boss,  Task  Performance  Judge  or  Competition 
Director. 

4.3.  Media  and  Visitor  Day 

A  media  and  visitor  day  was  conducted  on  Thursday  August  1 1,  201 1  to  provide  the  opportunity 
for  the  teams  to  describe  and  demonstrate  their  technologies  to  interested  parties  and  potential 
military  users.  Over  100  people  attended  the  events  including  representatives  from  the  Anny,  Air 
Force,  law  enforcement,  academia,  participating  companies,  elected  officials,  local  business 
leaders,  and  a  middle  school  Lego  Robotics  league  team.  Major  General  Luke  Reiner,  USA, 
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Adjutant  General  for  Wyoming  &  Commander  Wyoming  National  Guard,  was  the  ranking 
military  member  in  attendance.  BG  Harold  Reed,  USAF,  Chief  of  Staff  for  the  Wyoming  Air 
National  Guard  was  also  in  attendance.  Reporters  representing  regional  media  outlets  covered 
the  event  including:  K2TV  -  ABC  Affiliate  Casper,  Wyoming,  the  Platte  County  Records  Times, 
and  the  Lusk  Herald. 

The  teams  each  had  time  to  describe  their  systems  and  technologies  to  the  group  and  then 
demonstrate  the  systems  in  action.  The  teams  were  provided  an  area  to  demonstrate  all  the  events 
they  competed  in.  To  accommodate  the  demonstration,  3  previously  felled  trees  were  “planted” 
for  each  of  the  vegetation  clearance  competitors  to  cut  down.  This  was  accomplished  by  boring  a 
large  hole  with  an  auger  and  then  inserting  the  base  of  fallen  tree  in  to  the  hole  like  a  telephone 
pole  so  it  would  stand  upright.  After  the  demonstrations,  the  visitors  were  taken  on  a  window 
tour  of  the  actual  competition  sights  before  departing  (Figure  25  and  Figure  26) 


Team  UXOD  presentation  Team  UXOD  Demonstration 


Figure  25.  Media  &  Visitor  Day  Photos 
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Team  SKY  Research  presentation 


MG  Reiner  &  BG  Reed  talk  with  Mr.  Skibba 
and  Dr  Overholt 

Figure  26.  Media  &  Visitor  Day  Photos 


4.4.  Vegetation  Clearance  Event  Results 

4.4.1.  Competing  Teams 

The  two  teams  that  competed  in  the  Vegetation  Clearance  event  were  Team  D4C  and  Team 
UXOD  Automation.  Team  Sky  Research  did  not  participate  in  this  event. 

4.4.2.  Task  Description 

The  vegetation  clearance  task  of  the  R2C2  was  designed  to  simulate  a  real-world  UXO  clearance 
activity  as  closely  as  possible  and  allow  for  scoring.  Vegetation  clearance  is  typically  one  of  the 
initial  tasks  completed  to  prepare  a  site  for  the  surface  clearance,  geophysical  mapping,  and  sub¬ 
surface  clearance.  This  vegetation  typically  includes  all  trees,  bushes,  grass,  etc.  Typically,  for  a 
real  world  operation  the  vegetation  must  be  cut  low  enough  to  allow  for  the  final  intended  use  of 
the  range  or  to  enable  the  next  operation  in  the  range  clearance  process.  This  may  be  to  just 
provide  line  of  sight  to  a  target  or  to  provide  access  for  surface  clearance  and  geophysical 
mapping. 

For  the  purpose  of  this  competition,  the  trees  were  required  to  be  cut  down  to  8cm  or  less  in 
height  and  all  vegetation  debris  either  removed  from  the  sight  or  mulched  to  15  cm  or  less  in 
height.  The  event  score  was  determined  by  measuring  the  height  of  the  tree  stumps  and 
vegetation  residue  left  on  the  event  range.  Points  were  deducted  for  each  tree  or  stump  remaining 
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greater  than  8cm  above  the  surrounding  grade  and  any  remaining  vegetation  or  vegetation 
residue  greater  than  15cm  above  the  surrounding  grade. 

For  the  vegetation  clearance  task,  the  teams  had  eight  hours  to  clear  a  five-acre  site  of  50  trees 
and  mow  all  of  the  remaining  vegetation  down  to  a  height  of  15cm  or  less.  Tree  stumps  had  to  be 
8cm  or  lower  and  felled  trees  had  to  be  either  removed  from  the  site  or  mulched  in  place  to  less 
than  15cm’  depth. 

4.4.3.  Team  UXOD  Vegetation  Clearance 

Team  UXOD  Automation  conducted  the  vegetation  clearance  task  using  a  robotic  John  Deere 
Skid  Steer  and  feller  buncher.  These  normally  manned  systems  were  converted  to  remote  control 
using  a  Kairos  Autonomi  robotics  kit  with  Autonomous  Solutions  Mobius  system  being  used  for 
command  and  control. 

Team  UXOD  accomplished  this  task  by  using  a  combination  of  teleoperation  and  semi- 
autonomous  driving  using  both  their  skid  steer  w/grappling  attachment  and  feller  buncher.  An 
operator  would  teleoperate  the  machines  during  the  more  difficult  tasks  such  as  aligning, 
capturing  and  cutting  down  a  tree.  Then  the  robot  would  take  over  a  use  waypoint  driving  to 
semi-autonomously  carry  the  tree  back  to  a  collection  point  where  scripted  behaviors  were  used 
to  dump  the  tree  and  then  drive  back  to  the  next  tree  to  be  cut  (Figure  27).  The  skid  steer  was 
used  after  the  mowing  task  was  complete  during  the  vegetation  removal  operation. 


Figure  27.  Team  UXOD  Feller  Buncher  &  Skid  Steer  Vegetation  Removal 


For  the  mowing  task,  the  Team  UXOD  robotic  skid  steer  system  was  comprised  of  a  flail  mower 
mounted  on  the  front  to  grind  up  midsize  vegetation  such  as  shrubs  and  a  towed  array  of  mowers 
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attached  to  the  rear  to  cut  the  grass  and  remaining  vegetation  remnants.  The  skid  steer  was 
autonomously  controlled  using  a  Zamboni  pattern  to  cover  the  area  (Figure  28).  Team  UXOD 
Automation  successfully  cut  and  removed  all  the  trees  and  completed  the  mowing  task,  although 
their  large  feller  buncher  did  leave  some  heavy  ruts  which  cost  them  some  points  for  the  surface 
damage  category  (Figure  29). 


Figure  28.  Team  UXOD  Skid  Steer  Mowing  Task  Operation 


Figure  29.  Team  UXOD  Surface  Damage 


For  the  most  part,  their  system  of  systems  performed  very  well  and  they  efficiently  were  able  to 
mow  the  small  vegetation  and  cut  trees  simultaneously  running  both  the  robotic  skid  steer  and 
feller  buncher  at  the  same  time.  However,  they  did  have  some  difficulties  which  cost  them  time 
in  completion  of  the  task  to  recover  and  reset  their  systems. 

The  first  incident  was  when  the  feller  buncher  engaged  a  tree  too  hard  causing  the  top  half  of  the 
tree  to  break  and  fall  onto  the  top  of  the  feller  buncher  (Figure  30).  The  tree  dislocated  the 
communication  antenna  causing  the  system  to  lose  communications  and  cease  operations  per  the 
safety  design.  As  per  the  event  procedures  all  operations  on  this  range  were  halted  and  the  R2C2 
staff  inspected  the  system  to  make  sure  it  was  safe  to  allow  team  UXOD  to  initiate  recover  of  the 
system.  The  system  was  found  to  be  stable  and  team  UXOD  was  given  permission  to  remove  the 
tree,  fix  and  recover  the  feller  buncher  system.  The  event  clock  was  paused  during  the  R2C2 
staff  inspection  and  was  restarted  as  soon  as  the  team  was  notified  that  they  could  proceed  with 
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the  recovery  operations.  The  damage  to  the  system  was  minor  and  was  fixed  by  team  UXOD 
replacing  the  antenna  cable  and  reattaching  it  to  the  machine  (Figure  31). 


Figure  30.  Team  UXOD  Feller  Buncher  Loss  Communications  Incident 


Figure  31.  Team  UXOD  Communication  Antenna  Repair 


The  second  incident  occurred  when  Team  UXOD’s  robotic  skid  steer  was  using  the  flail  mower 
to  mow  the  hilly  area  between  the  trees  (Figure  32).  The  system  was  being  teleoperated  at  the 
time  and  the  operator  was  driving  downhill  with  the  flail  in  the  raised  position.  The  system  tilted 
forward  and  came  to  rest  against  the  flail  due  to  the  forward  center  of  gravity  that  this 
configuration  imposed  on  the  system.  They  were  unable  to  upright  the  system  remotely  so  the 
event  was  halted  for  a  second  time  for  the  R2C2  staff  to  determine  that  the  situation  was  safe  and 
stable.  Again  the  situation  was  safe,  the  event  clock  was  restarted  and  team  UXOD  given 
permission  to  recover  the  vehicle.  They  manually  used  their  feller  buncher  system  to  pull  the 
skid  steer  upright  so  a  technician  could  enter  the  cab  and  lower  the  flail.  Both  systems  were  reset 
to  their  approximate  positions  when  the  event  happened  and  robotic  operations  were  allowed  to 
resume  once  all  personnel  were  safely  off  the  range  and  in  the  designated  crew  or  observer  areas. 
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Figure  32.  Team  UXOD  John  Deer  Skid  Steer  Roll-over  Incident 


Team  UXOD  used  the  entire  8hrs  to  complete  this  task  and  exceeded  their  allotted  man-hours  by 
8  hours  since  their  program  manager  joined  the  crew  to  assist  in  the  recovery  operations.  They 
achieved  a  total  score  of  158.62  out  a  possible  250  points.  Detailed  scoring  sheets  are  shown  in 
Table  3. 

Overall  their  systems  did  a  good  job  of  achieving  the  event  goals  of  cutting  trees  to  less  than  8cm 
in  height  and  mowing  the  vegetation  and  vegetation  residue  to  less  than  15cm  in  height.  The 
feller  buncher  system  was  very  fast  and  efficient,  but  it  was  difficult  for  them  to  quickly  cut  the 
stumps  to  less  than  8cm  in  height.  The  feller  buncher  had  skids  that  kept  the  cutter  head  10cm  off 
the  ground  (Figure  33).  In  order  to  achieve  the  required  stump  height  they  had  to  tilt  the  head  up 
at  an  angle  and  dig  the  skids  into  the  ground  on  both  sides  of  the  tree.  This  slowed  the  operation 
significantly  and  cost  them  time  to  finish  the  task.  Their  combination  flail  mower  in  front  of  the 
skid  steer  towing  an  array  of  finish  mowers  behind  did  an  excellent  job  of  mowing  the  grass  and 
brush  areas.  The  flail  mower  broke  up  all  the  large  items  and  rocks  and  the  finish  mowers  cut  the 
remaining  residue  and  vegetation  to  a  consistent  height.  The  system  was  operated  entirely  in 
GPS  waypoint  mode  for  this  part  of  the  task  and  performed  very  well  driving  straight  consistent 
patterns  over  the  subject  area. 


Figure  33.  Team  UXOD  Feller  BuncherTree  Stump  Cut 
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Table  3.  Team  UXOD  Vegetation  Clearance  Scores 


Team  UXOD  Automation 

Vegetation  Clearance  Event  Score 

158.62  out  a  possible  250  points 

Metric  Categories 

Total  for  each  Metric 

Weight 

System  Task  Performance 

203.83 

50% 

Level  of  Human  Interaction 

91.75 

40% 

Man  Hours  to  Perform  the  Task 

200.00 

10% 

Competition  Level  Metrics 

Points/Violation 

Violations 

Total 

Perimeter  or  No-go  area  violation 

50 

0 

0 

Exclusion  zone  violation 

DQ 

Surface  (>15cm) 

2 

8 

16 

Surface  grade  (>15cm) 

0 

0 

Total  Penalty 

16 

System  Task  Performance  (250)  Points 

203.83 

Possible  Points 

Vegetation  Tree  Removal  (>8cm) 

115.25 

Vegetation  Residue  Removal  (>15cm) 

104.58 

Level  of  Human  Interaction  (250) 

91.75 

Possible  Points 

(from  Human  Interaction  Judge) 

Description  of  Level  of  Human  Interaction 

250 

Near  zero  human  interaction 

100% 

63% 

Minimal  interaction 

75% 

Moderate  Interaction 

50% 

Frequent  Interaction 

25% 

Continuous  Interaction  (completely  tele-operated) 

0% 

Man  Hours  to  Perform  the  Task  (250)  Points 

200.00 

Start  Time 

End  Time 

Elapsed  Time 

8:00:00 

8:00:00 

16:00:00 

Number  of  people  on  site 

5 

Total  Man-Hours 

40.00 

Man  Hours  Allotted 

32 

4.4.4.  Team  D4C  Vegetation  Clearance 

Team  D4C  used  two  Bobcat  skid  steer  loaders  controlled  by  the  QinetiQ  Robotic  Control  Kits 
for  the  vegetation  clearance  task.  They  employed  a  tree  shear  on  one  system  while  the  second 
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system  used  a  brush  mower  over  the  non-treed  area  (Figure  34  and  Figure  35).  Like  Team 
UXOD  Automation,  Team  D4C  used  teleoperation  for  the  more  difficult  chores,  such  as 
engaging  and  cutting  the  trees  and  autonomous  waypoint  driving  for  the  simpler  tasks  of  mowing 
the  open  areas  and  travel  to  and  from  the  collection  area.  Team  D4C  successfully  cut  and 
removed  all  the  trees  to  the  specification  and  completed  the  mowing  task  while  leaving  no 
damage  on  the  site. 


Figure  34.  Team  D4C  Bobcat  Tree  Sheer 


Figure  35.  Team  D4C  Bobcat  Mower 


The  Team  D4C  solution  was  very  efficient  and  smooth.  The  operators  seemed  to  be  well 
practiced  and  was  extremely  efficient  in  cutting  down  the  trees  and  carrying  them  to  the 
collection  pile.  The  process  they  used  was  to  drive  to  a  spot  near  the  tree  using  waypoint 
guidance,  switch  to  teleoperation  and  have  the  operator  engage  the  tree  and  cut  it  with  the  shear. 
After  the  tree  was  felled,  the  operator  used  the  open  shear  to  straddle  and  grab  the  butt  of  the  tree 
and  then  either  teleoperate  or  waypoint  drive  the  system  to  the  collection  pile  and  dump  the  tree. 

While  this  was  being  done,  the  second  Bobcat  robotics  skid  steer  was  performing  the  mowing 
task  using  a  rough  cut  field  mower  on  the  front  of  the  skid  steer  system.  The  mowing  was 
accomplished  using  GPS  waypoint  driving.  The  rough  cut  mower  did  not  cut  as  cleanly  as  the 
Team  UXOD  finish  mower,  but  its  final  product  was  sufficient  to  meet  the  requirement.  Once 
the  mowing  task  was  complete,  the  mower  deck  was  removed  and  a  grapple  attachment  placed 
on  the  2nd  Bobcat  skid  steer.  This  allowed  the  team  to  speed  up  the  tree  cutting  process  as  the  tree 
shear  equipped  robot  could  concentrate  on  just  cutting  trees  and  the  grapple  robot  could  move 
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them  to  the  collection  pile  (Figure  36).  This  required  a  2nd  operator  to  teleoperate  the  system 
during  the  grapple  engagement  and  dumping.  The  travel  to  the  pile  was  still  completed  using 
GPS  waypoint  driving. 

After  all  trees  were  removed,  a  mulching  head  was  attached  to  the  1st  Bobcat  and  the  mower 
deck  was  put  back  on  the  2nd  Bobcat  and  then  used  to  clear  midsize  vegetation  such  as  small 
trees  and  shrubs,  and  mow  in  between  the  tree  areas  until  the  time  had  expired,  respectively 
(Figure  37).  Team  D4C  had  no  incidents  which  caused  them  to  have  to  recover  their  systems. 
They  also  used  the  entire  8  hrs  to  complete  the  task  but  did  not  exceed  their  allotted  man-hours. 
They  achieved  a  total  score  of  177.38  out  a  possible  250  points.  Detailed  scoring  sheets  are 
shown  in  Table  4. 


Figure  36.  Team  D4C  Bobcat  Grapple 


Figure  37.  Team  D4C  Bobcat  Midsize  Vegetation  Removal  and  Mowing 
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Table  4.  Team  D4C  Vegetation  Clearance  Scores 


Team  D4C 

Vegetation  Clearance  Event  Score 

177.38  out  a  possible  250  points 

Metric  Categories 

Total  for  each  Metric 

Weight 

System  Task  Performance 

235.17 

50% 

Level  of  Human  Interaction 

87.00 

40% 

Man  Hours  to  Perform  the  Task 

250 

10% 

5 

Competition  Level  Metrics 

Points/Violation 

Violations 

Total 

Perimeter  or  No-go  area  violation 

50 

0 

0 

Exclusion  zone  violation 

DQ 

Surface  (>15cm) 

0 

0 

0 

Surface  grade  (>15cm) 

0 

0 

Total  Penalty 

0 

System  Task  Performance  (250)  Points 

235.17 

Possible  Points 

Vegetation  Tree  Removal  (>8cm) 

123.50 

Vegetation  Residue  Removal  (>15cm) 

111.67 

Level  of  Human  Interaction  (250) 

87.00 

Possible  Points 

(from  Human  Interaction  Judge 

Description  of  Level  of  Human  Interaction 

250 

Near  zero  human  interaction 

100% 

65% 

Minimal  interaction 

75% 

Moderate  Interaction 

50% 

Frequent  Interaction 

25% 

Continuous  Interaction  (completely  tele-operated) 

0% 

Man  Hours  to  Perform  the  Task  (250)  Points 

250.00 

Start  Time 

End  Time 

Elapsed  Time 

8:00:00 

8:00:00 

16:00:00 

Number  of  people  on  site 

4 

Total  Man-Hours 

32.00 

Man  Hours  Allotted 

32 

4.4.5.  Comments 

The  following  comments  are  based  from  the  event  judge  Walt  Waltz: 

1)  The  flail  mower  used  by  UXOD  was  not  very  effective  when  used  alone  in  this  type 
vegetation.  The  west  surface  area  was  mowed  exceptionally  well  where  the  flail  mower 
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was  used  in  conjunction  with  lawn  mowers.  The  only  damage  occurred  in  grid  29  was  a 
result  of  the  feller/buncher’s  cutting  head  contacting  the  ground  while  moving. 

2)  The  feller  buncher  was  used  in  a  downhill  cutting  approach  resulting  in  a  tipping  of  the 
head  to  try  to  achieve  the  lowest  cut  possible.  It  would  be  interesting  to  see  how  the  cut 
would  be  from  an  uphill  perspective.  When  the  cuts  were  made,  there  appeared  to  be 
significant  vehicle  travel  after  the  cut  was  made  due  to  momentum  or  possible  reaction 
time  of  the  operator.  An  uphill  cut  might  be  more  controllable. 

3)  The  skid  steer  behavior  after  initiating  auto-travel  mode  was  very  oscillatory  often 
sweeping  the  vehicle  very  severely.  Further  work  to  smooth  this  motion  is  needed. 

4)  The  footprint  of  the  track  vehicles  used  rendered  very  little  surface  damage  in  either 
competitor’s  areas,  especially  team  D4C  as  no  surface  damage  was  measured.  The 
majority  of  the  surface  damage  was  a  result  of  the  large  feller  buncher  tires. 

Overall: 

Team  UXO-D  got  off  to  a  slow  start  and  didn’t  appear  they  would  achieve  the  objective  of 
removing  all  50  trees.  Only  four  to  five  trees  were  removed  in  the  first  two  hours  where  team 
D4C  had  over  20  trees  cut  and  felled  but  not  yet  removed  from  the  operational  area.  It  was  clear 
that  UXO-D ’s  operator  was  improving  significantly  by  cutting  and  removing  two  or  three  trees  at 
a  time  and  made  up  significant  time  by  mid  afternoon.  It  was  also  obvious  that  team  D4C’s 
operator  was  very  skilled  and  his  methodology  of  cutting,  falling  and  dragging  his  trees  to  the 
top  of  the  hill  was  very  effective.  A  significant  amount  time  was  used  with  the  feller  buncher  in 
traversing  the  cut  trees  up  over  the  hill  to  the  designated  dumping  area.  This  might  have  been 
sped  up  using  their  skid-steer  (V2)  more  in  the  removal  process  with  the  grapple  bucket. 

Using  the  flail-mower  without  the  other  mowers  also  didn’t  do  very  well  on  the  grassy 
vegetation.  An  alternative  attachment  like  the  mulching  one  used  by  D4C  would  have  been  more 
effective  reducing  stump  height  and  smoothing  the  surface  damage.  D4C  spent  significant  time 
to  mulch  and  back-drag  their  entire  hill  area  which  resulted  in  no  violation  points  for  surface 
damage  and  only  two  minor  tree-stump  violations. 

4.5.  Surface  Clearance  Event  Results 

4.5.1.  Competing  Teams 

The  two  teams  that  competed  in  the  Surface  Clearance  event  were  Team  D4C  and  Team  UXOD 
Automation.  Team  Sky  Research  did  not  participate  in  this  event. 

4.5.2.  Task  Description 

The  surface  clearance  task  of  the  R2C2  was  designed  to  simulate  surface  debris  removal  from 
target  impact  zones.  For  the  purpose  of  the  competition  only  ferrous  debris  were  used  to  simulate 
typical  artillery  and  mortar  ordnance  scrap  and  allow  for  effective  scoring.  On  a  typical  target 
range  there  is  a  great  amount  of  ordnance  scrap,  in  addition  to  the  UXO  items  left  on  the  surface 
that  must  be  removed  prior  to  surveying  the  area  with  detection  systems.  This  scrap  can  be  from 
a  variety  of  sources  including:  range  targets,  munitions  debris/fragmentation,  and  unexploded 
ordnance  (UXO).  A  large  part  of  the  debris  field  consists  of  ferrous  metal  objects  that  magnetic 
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collection  devices  will  retrieve,  however  most  real  world  sites  contain  non-ferrous  materials  as 
well.  For  the  competition,  the  surface  field  was  only  seeded  with  only  ferrous  items  of  interest 
(IOI),  including  two  fixed  obstacles  to  simulate  targets  like  tanks,  vehicles,  etc. 

The  purpose  of  this  event  was  for  each  team  to  collect  as  many  of  the  seeded  ferrous  objects  as 
possible  and  deposit  them  at  the  designated  collection  area.  The  collection  piles  were  scored  for 
pile  cleanliness  as  well  as  the  number  of  items  recovered.  Therefore  it  was  important  that  the 
teams  chosen  method  minimized  the  collection  of  non-scrap  objects  such  as  vegetation,  soil,  and 
rocks.  This  is  due  to  the  fact  the  explosive  ordnance  disposal  technicians  have  to  go  through  the 
collected  debris,  and  it  is  much  more  dangerous  if  the  items  have  to  be  separated  from 
contamination  such  as  dirt  and  vegetation. 

The  IOIs  consisted  of  a  mix  of  two  pipe  diameters,  three  angle  iron  types  and  two  sizes  of  flat 
plate.  550  IOIs  were  randomly  deposited  throughout  the  range  area  and  a  few  were  randomly 
depressed  into  the  soil  (Figure  38).  Competitors  could  use  a  variety  of  techniques  such  as 
magnets,  surf  rakes,  etc  to  collect  these  items  and  transport  them  to  the  designated  collection 
area. 


Figure  38.  Industry  Standard  Objects  of  Interest 


4.5.3.  Team  UXOD  Surface  Clearance 

Team  UXOD  Automation  once  again  employed  a  multiple  robotic  solution  for  the  surface 
clearance  task.  They  used  their  Kubota  UTV  to  tow  a  permanent  magnet  trailer  and  the  skid  steer 
to  carry  the  electromagnet  system  and  generator.  Both  systems  used  waypoint  driving  and 
scripted  behaviors  for  the  majority  of  this  work  (Figure  39  and  Figure  40). 
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Figure  39.  Team  UXOD  Automation  Robotic  Solutions  for  Surface  Clearance 
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Team  UXOD  started  with  just  the  Kubota  UTV  operating  until  it  had  several  passes  complete  to 
get  a  head  start  and  some  clearance  from  the  2nd  robot  the  John  Deere  skid  steer  with  the 
electromagnet.  They  used  a  down  and  back  modified  zamboni  pattern.  This  approach  was  very 
successful  eventually  recovering  393  of  550  items  (Figure  41).  The  electromagnet  was  much 
more  effective  at  collecting  and  holding  the  items  than  the  weaker  permanent  magnet  system. 
However,  their  two-robot  solution  worked  against  them  as  the  second  robot  -the  skid  steer  - 
drove  over  the  items  the  first  robot  left  in  the  collection  area,  causing  them  to  lose  many  points 
for  pile  cleanliness.  They  achieved  a  total  score  of  161.86  out  of  a  possible  250  points.  Detailed 
scoring  sheets  are  shown  in  Table  5. 


Figure  41.  Team  UXOD  Collection  Piles  of  Surface  Clearance  Task: 
Clean  Collection  Pile  (top);  Buried  IOI’s  (bottom) 


In  addition  they  had  one  incident  that  cost  them  time  when  their  Kubota  UTV  robot  system  drove 
into  a  natural  swale  that  was  part  of  their  surface  range  area.  The  operators  monitoring  the 
system  became  alarmed  and  e-stopped  the  system  when  they  observed  the  system  rapidly  tip 
nose  down  as  it  entered  the  swale.  They  tried  to  teleoperate  the  system  to  back  out  of  the  swale 
but  could  not  exit  that  way  as  the  system  could  not  get  enough  traction.  Eventually,  they 
teleoperated  the  system  through  the  swale  and  up  and  out  the  other  side  where  it  resumed  its 
course  and  waypoint  driving  pattern. 

It  should  be  noted  that  all  teams  had  two  hours  prior  to  the  start  of  the  event  to  walk  the  range 
and  note  any  features  to  incorporate  into  their  plan.  They  were  also  provided  detailed  map 
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coordinates  for  their  event  areas  two  weeks  prior  to  the  event.  Team  UXOD  did  not  take 
sufficient  advantage  of  this  time  and  map  data  to  plan  for  the  natural  features  so  their  system’s 
would  not  encounter  problems  with  terrain  that  they  could  not  handle.  In  addition,  all  teams  were 
encouraged  to  incorporate  obstacle  detection  and  avoidance  technologies  and  choose  not  to  do 
so.  On  real  world  UXO  ranges  the  terrain  is  composed  of  many  impact  craters  and  irregularities 
from  the  ordnance  items  that  must  be  planned  for  or  negotiated  by  systems  with  suitable 
mobility. 


Table  5.  Team  UXOD  Surface  Clearance  Scores 


Team  UXOD 

Surface  Clearance  Event  Score 

161.86  out  a  possible  250  points 

Metric  Categories 

Total  for  each  Metric 

Weight 

System  Task  Performance 

154.3 

50% 

Level  of  Human  Interaction 

149.25 

40% 

Man  Hours  to  Perform  the  Task 

5.  250 

6.  10% 

Competition  Level  Metrics 

Points/Violation 

Violations 

Total 

Perimeter  or  No-go  area  violation 

50 

0 

0 

Exclusion  zone  violation 

DQ 

Surface  (>15cm) 

0 

0 

0 

Surface  grade  (>15cm) 

0 

0 

Total  Penalty 

0 

System  Task  Performance  (250)  Points 

154.32 

Possible  Points 

Score 

number  of  seeded 
items 

number 

collected 

Percent  of  debris  removed 

125 

89.32 

550 

393 

Pile  cleanliness 

125 

65.00 

Level  of  Human  Interaction  (250) 

149.25 

Possible  Points 

(from  Human  Interaction  Judge) 

Description  of  Level  of  Human  Interaction 

250 

Near  zero  human  interaction 

100% 

40.3% 

Minimal  interaction 

75% 

Moderate  Interaction 

50% 

Frequent  Interaction 

25% 

Continuous  Interaction  (completely  tele-operated) 

0% 

Man  Hours  to  Perform  the  Task  (250)  Points 

250.00 

Elapsed  Time 

8:00:00 
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Number  of  people  on  site 

5 

Total  Man-Hours 

40.00 

Man  Hours  Allotted 

40 

6.1.1.  Team  D4C  Surface  Clearance 

Team  D4C  employed  just  one  of  their  robotic  Bobcat  systems  with  a  pennanent  magnet  mounted 
to  the  skid  plate  to  sweep  the  site  (Figure  42).  This  task  was  completely  waypoint  driven  with 
scripted  behaviors  for  the  dumping  of  the  collected  items.  They  used  a  “U”  shaped  pattern  that 
was  very  effective  for  minimizing  travel  distance  and  allowing  them  to  cleanly  dump  their  debris 
at  both  ends  of  the  pattern.  It  was  important  to  minimize  carrying  the  collected  items  any  further 
than  needed  as  the  terrain  or  machine  vibrations  could  knock  off  some  of  the  items  from  the 
magnet  face.  Their  permanent  magnet  only  solution  was  not  as  effective  as  the  electromagnet 
that  Team  UXOD  used,  but  team  D4C  completed  the  sweep  in  less  than  the  full  eight  hours  and 
recovered  243  of  the  550  items  with  very  clean  debris  piles  (Figure  43). 


Figure  42.  Team  D4C  Bobcat  Solution  for  Surface  Clearance 
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They  did  not  have  any  interruptions  once  the  event  started  and  the  system  ran  well  on  its  own 
with  just  monitoring  by  the  operators  through  the  whole  operation.  They  achieved  a  total  score  of 
191.91  out  of  a  possible  250  points.  This  is  due  to  half  of  the  score  for  both  the  surface  and 
subsurface  clearance  tasks  were  for  collecting  all  of  the  seeded  items,  and  the  other  half  was  for 
clean  debris  piles.  Detailed  scoring  sheets  are  shown  in  Table  6. 


Table  6.  Team  D4( 

2  Surface  Clearance  Scores 

Surface  Clearance  Event  Score 

191.91  out  a  possible  250  points 

Metric  Categories 

Total  for  each  Metric 

Weight 

System  Task  Performance 

178.23 

50% 

Level  of  Human  Interaction 

194.5 

40% 

Man  Hours  to  Perform  the  Task 

250.00 

10% 

Competition  Level  Metrics 

Points/Violation 

Violations 

Total 

Perimeter  or  No-go  area  violation 

50 

0 

0 

Exclusion  zone  violation 

DQ 

Surface  (>15cm) 

0 

0 

0 

Surface  grade  (>15cm) 

0 

0 

Total  Penalty 

0 

System  Task  Performance  (250)  Points 

178.23 

Possible  Points 

Score 

number  of  seeded 
items 

number 

collected 

Percent  of  debris  removed 

125 

55.23 

550 

243 

Pile  cleanliness 

125 

123.00 

Level  of  Human  Interaction  (250) 

194.50 

Possible  Points 

(from  Human  Interaction  Judge) 

Description  of  Level  of  Human  Interaction 

250 

Near  zero  human  interaction 

100% 

22.20% 

Minimal  interaction 

75% 
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Moderate  Interaction 

50% 

Frequent  Interaction 

25% 

Continuous  Interaction  (completely  tele-operated) 

0% 

Man  Flours  to  Perform  the  Task  (250)  Points 

250.00 

Elapsed  Time 

8:00:00 

Number  of  people  on  site 

3 

Total  Man-Flours 

24.00 

Man  Flours  Allotted 

24 

6.1.2.  Comments 

The  following  comments  are  based  from  the  event  judge  Walt  Waltz: 

1)  This  competition  was  made  extremely  simple  from  a  real-world  application.  Most 
areas  would  be  littered  throughout  with  a  mix  of  ferrous/non-ferrous  material  along  with 
a  host  of  other  debris.  It  would  have  been  more  practical  to  require  competitors  to  gather 
more  than  just  ferrous  items  that  would  require  very  different  techniques  and  that  would 
be  much  more  aggressive  than  magnetic  pick  up  devices. 

2)  D4C’s  method  of  dumping  in  the  collection  area  resulted  in  point  deductions  for  pile 
cleanliness  due  to  the  fact  that  the  rotation  caused  some  IOI’s  to  be  covered  from  the 
track  mounding  up  soil.  Also  executing  pivot  turns  at  the  end  of  each  run  resulted  in 
losing  IOIs  off  the  magnet. 

3)  Team  UXO-D  employed  a  combination  of  techniques,  one  of  which  (the 
electromagnet)  proved  to  be  very  effective  for  removing  large  ferrous  items  without 
rolling  off  the  magnet. 

4)  Team  UXO-D  used  a  lot  of  autonomous  operation  and  to  do  so,  ran  a  pattern  that  took 
them  over  the  top  of  previously  dumped  IOIs.  By  running  this  heavy  equipment  over  the 
top  of  UXO  and  twisting/turning  resulted  in  burying  several  IOIs.  This  resulted  in  a 
significant  deduction  for  creating  a  contaminated  area  in  the  collection  area. 

6.2.  Geophysical  Mapping  Event  Results 
6.2.1.  Competing  Teams 

All  three  teams  (Team  D4C,  Team  UXOD  Automation,  and  Team  Sky  Research)  competed  in 
the  Geophysical  Mapping  event. 
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6.2.2.  Task  Description 

The  geophysical  mapping  challenge  was  intended  to  demonstrate  a  robotic  system  navigating  a 
digital  geophysical  mapping  platfonn  towing  a  geophysical  sensor  array  within  the  designated 
area.  The  system  was  to  tow  a  time  domain  electromagnetic  induction  metal  detector  and  record 
its  data  over  100%  of  the  designated  area  at  a  line  spacing  of  50cm.  The  positional  accuracy  of 
the  digital  geophysical  should  have  been  sufficient  to  detect  and  locate  buried  metallic  objects  to 
30cm  or  better.  The  collected  raw  geophysical  data  should  have  been  at  or  below  an  objective 
noise  level  determined  at  the  site. 

For  the  geophysical  mapping  task,  the  teams  also  had  eight  hours  to  provide  the  most  accurate 
map  of  the  buried  items  while  maintaining  consistent  speed  and  control  of  the  system  to 
maximize  the  performance  of  the  sensor.  The  system  also  had  to  deal  with  some  simulated 
obstacles  meant  to  represent  blown-up  targets  that  would  be  found  on  real  target  ranges. 

For  all  teams,  the  geophysical  mapping  task  was  the  most  automated,  as  the  systems  just  had  to 
tow  a  precision  path  across  the  field  while  avoiding  the  two  obstacles.  Teams  were  given 
coordinates  of  the  competition  areas  and  obstacles  two  weeks  prior  to  the  competition  to  simulate 
a  priori  data  available  to  real-world  operations.  Some  teams  did  manual  path  planning  while 
others  had  automated  path  planning  as  part  of  their  control  solution 

6.2.3.  Team  D4C  Geophysical  Mapping 

Team  D4C  conducted  the  geophysical  mapping  task  using  a  Bobcat  Skid  Steer  with  a  QinetiQ 
North  America  robotics  kit  to  tow  an  electromagnetic  sensor  over  the  1 .6  acre  site  that  was 
seeded  with  simulated  buried  UXO  (Figure  44).  The  teams  original  path  plan  did  not  incorporate 
the  obstacles  due  to  an  oversight  by  their  programmers.  To  accommodate  the  obstacles,  they 
implemented  a  diagonal  pattern  that  purposely  avoided  large  strips  of  the  range.  This  pattern 
caused  them  to  use  a  large  portion  of  their  travel  time  in  the  turns  and  not  scanning  the  range 
areas  of  interest.  This  task  was  completed  with  almost  no  human  interaction,  as  the  system 
conducted  waypoint  driving  over  the  course  to  complete  the  mapping  task,  finding  92  of  the  125 
items.  They  achieved  a  total  score  of  165.30  out  of  a  possible  250  points.  Detailed  scoring  sheets 
are  shown  in  Table  7. 

Team  D4C  acquired  geophysical  mapping  data  for  58.41%  of  the  total  coverage  area  (Figure  45) 
and  accurately  located  92  targets  (Figure  46). 


Figure  44.  Team  D4C  Geophysical  Mapping  Task 


46 

Distribution  A:  Approved  for  public  release;  distribution  unlimited.  88ABW-2012-5009,  19  September  2012. 


516260  516260  516300  516320  516340  516360  516360  516400 


MAP 

Footprint  Coverage 


LEGEND 


Arco  covered  3*40  59  m» 
Totai  area  6404.57  m* 
Pe*cortago  covered  58  41  % 
inttrwnont  fool  pivii;  053  m 


Figure  45.  Team  D4C  Geophysical  Coverage  Map 


Figure  46.  Anomaly  Location  Accuracy  for  Team  D4C 
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Table  7.  Team  D4C  Geophysical  Mapping  Scores 


Team  D4C 

Surface  Clearance  Event  Score 

165.30  out  a  possible  250  points 

Metric  Categories 

Total  for  each  Metric 

Weight 

System  Task  Performance 

174.00 

50% 

Level  of  Human  Interaction 

133.25 

40% 

Man  Hours  to  Perform  the  Task 

250.00 

10% 

Competition  Level  Metrics 

Points/Violation 

Violations 

Total 

Perimeter  or  No-go  area  violation 

50 

0 

0 

Exclusion  zone  violation 

DQ 

Surface  (>15cm) 

0 

0 

Surface  grade  (>15cm) 

0 

0 

Total  Penalty 

0 

System  Task  Performance  (250)  Points 

174.00 

Possible  Points 

Violations 

Total 

Delivery  of  Raw  Data  (correct  format  within 
48hrs) 

250 

0 

0 

Noise  Level 

70 

70 

Sensor  Coverage 

60 

39 

Anomaly  Location  Accuracy 

60 

45 

Survey  Speed 

60 

20 

Level  of  Human  Interaction  (250) 

133.25 

Possible  Points 

(from  Human  Interaction  Judge) 

Description  of  Level  of  Human  Interaction 

250 

Near  zero  human  interaction 

100% 

46.7% 

Minimal  interaction 

75% 

Moderate  Interaction 

50% 

Frequent  Interaction 

25% 

Continuous  Interaction  (completely  tele-operated) 

0% 

Man  Hours  to  Perform  the  Task  (250)  Points 

250.00 

Elapsed  Time 

8:00:00 

Number  of  people  on  site 

3 

Total  Man-Hours 

24.00 

Man  Hours  Allotted 

24 
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6.2.4.  Team  Sky  Research  Geophysical  Mapping 

Team  Sky  Research  only  entered  in  the  geophysical  mapping  event.  Their  robotic  solution  used  a 
Kubota  UTV  outfitted  with  a  Topcon  autosteering  system  for  waypoint  driving  and  steering 
control  towing  a  Sky  Research  sensor  (Figure  47).  Sky  Research  personnel  converted  the  Kubota 
and  adapted  the  Topcon  system  for  use  as  an  independent  robotic  solution,  since  the  Topcon  is 
normally  intended  for  use  to  augment  a  manned  machine.  The  team  had  many  software  problems 
during  the  day  that  slowed  their  progress.  They  spent  a  great  deal  of  time  troubleshooting  and 
trying  to  implement  fixes  to  their  system  to  complete  the  course.  Due  to  these  problems  they 
were  only  able  to  complete  about  half  of  the  mapping  course,  finding  69  of  the  125  items.  They 
achieved  a  total  score  of  133.30  out  of  a  possible  250  points.  Detailed  scoring  sheets  are  in  Table 


8. 


Figure  47.  Team  Sky  Research  Geophysical  Mapping  Task 


Team  Sky  Research  acquired  geophysical  mapping  data  for  40.06%  of  the  total  coverage  area 
(Figure  48)  and  accurately  located  69  targets  (Figure  49). 


Figure  48.  Team  Sky  Research  Geophysical  Coverage  Map 
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Figure  49.  Anomaly  Location  Accuracy  Analysis  for  Sky  Research 


Table  8.  Team  Sky  Research  Geophysical  Mapping  Scores 


Surface  Clearance  Event  Score 

133.3  out  a  possible  250  points 

Metric  Categories 

Total  for  each  Metric 

Weight 

System  Task  Performance 

102.00 

50% 

Level  of  Human  Interaction 

143.25 

40% 

Man  Hours  to  Perform  the  Task 

250.00 

10% 

Competition  Level  Metrics 

Points/Violation 

Violations 

Total 

Perimeter  or  No-go  area  violation 

50 

1 

50 

Exclusion  zone  violation 

DQ 

Surface  (>15cm) 

0 

0 

Surface  grade  (>15cm) 

0 

0 

Total  Penalty 

50 

System  Task  Performance  (250)  Points 

102.00 

Possible  Points 

Violations 

Total 

Delivery  of  Raw  Data  (correct  format  within 
48hrs) 

250 

0 

0 

Noise  Level 

70 

70 

Sensor  Coverage 

60 

28 

Anomaly  Location  Accuracy 

60 

34 

Survey  Speed 

60 

20 
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Level  of  Human  Interaction  (250) 

143.25 

Possible  Points 

(from  Human  Interaction  Judge) 

Description  of  Level  of  Human  Interaction 

250 

Near  zero  human  interaction 

100% 

42.7% 

Minimal  interaction 

75% 

Moderate  Interaction 

50% 

Frequent  Interaction 

25% 

Continuous  Interaction  (completely  tele-operated) 

0% 

Man  Hours  to  Perform  the  Task  (250)  Points 

250.00 

Elapsed  Time 

8:00:00 

Number  of  people  on  site 

3 

Total  Man-Hours 

24.00 

Man  Hours  Allotted 

24 

6.2.5.  Team  UXOD  Geophysical  Mapping 

Team  UXOD  Automation  used  their  robotic  Kubota  UTV  towing  a  Zonge  electromagnetic 
sensor  system  (Figure  50).  As  with  all  of  UXOD’s  systems,  the  Kairos  kit  provided  the  robotic 
control  of  the  UTV  with  ASI’s  Mobius  being  used  for  the  path  planning  and  waypoint  control. 
The  UXOD  system  was  very  stable  and  performed  a  perfectly  boring  job  over  the  course, 
successfully  mapping  all  125  seeded  items.  Approximately  halfway  through  the  event  Team 
UXOD  halted  their  system  to  do  a  manual  download  of  their  data  to  that  point.  Sometime  after 
restarting  this  download,  their  system  quit  logging  the  geophysical  mapping  data.  Fortunately  for 
them,  all  of  the  lost  mapping  data  was  in  an  area  designated  for  ground  truth  and  noise 
measurements  so  it  contained  no  simulated  UXO  items.  The  missing  data  can  be  clearly  seen  in 
their  map  plot  in  Figure  52.  They  achieved  a  total  score  of  205.70  out  of  a  possible  250  points. 
Detailed  scoring  sheets  are  shown  in  Table  9.  Team  UXOD  acquired  geophysical  mapping  data 
for  83.96%  of  the  total  coverage  area  (Figure  51)  and  accurately  located  all  125  targets  (Figure 
52). 


Figure  50.  Team  UXOD  Geophysical  Mapping  Task 
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Figure  51.  Team  UXOD  Geophysical  Coverage  Map 


Figure  52.  Anomaly  Location  Accuracy  for  Team  UXOD 
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Table  9.  Team  UXOD  Geophysical  Mapping  Scores 


Team  UXOD 

Geophysical  Mapping  Event  Score 

205.7  out  a  possible  250  points 

Metric  Categories 

Total  for  each  Metric 

Weight 

System  Task  Performance 

189.00 

50% 

Level  of  Human  Interaction 

215.5 

40% 

Man  Hours  to  Perform  the  Task 

250.00 

10% 

Competition  Level  Metrics 

Points/Violation 

Violations 

Total 

Perimeter  or  No-go  area  violation 

50 

0 

0 

Exclusion  zone  violation 

DQ 

Surface  (>15cm) 

0 

0 

Surface  grade  (>15cm) 

0 

0 

Total  Penalty 

0 

System  Task  Performance  (250)  Points 

189.00 

Possible  Points 

Violations 

Total 

Delivery  of  Raw  Data  (correct  format  within 
48hrs) 

250 

0 

0 

Noise  Level 

70 

70 

Sensor  Coverage 

60 

54 

Anomaly  Location  Accuracy 

60 

60 

Survey  Speed 

60 

5 

Level  of  Human  Interaction  (250) 

215.50 

Possible  Points 

(from  Human  Interaction  Judge) 

Description  of  Level  of  Human  Interaction 

250 

Near  zero  human  interaction 

100% 

75.0% 

Minimal  interaction 

75% 

Moderate  Interaction 

50% 

Frequent  Interaction 

25% 

Continuous  Interaction  (completely  tele-operated) 

0% 

Man  Hours  to  Perform  the  Task  (250)  Points 

250.00 

Elapsed  Time 

8:00:00 

Number  of  people  on  site 

5 

Total  Man-Hours 

40.00 

Man  Hours  Allotted 

40 
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6.2.6.  Comments 

Geophysical  data  from  the  three  competitors  were  analyzed  following  the  methods  outlined  in 
the  final  R2C2  Rules  and  Metrics  document.  All  data  analysis  was  performed  in  the  same 
manner  for  each  of  the  three  competitor’s  data.  All  scores  were  detennined  using  the  processes 
described  in  the  R2C2  Rules  and  Metrics  document.  Refer  to  Appendix  D  for  further  details  and 
the  in  depth  analysis  for  scoring  of  the  geophysical  mapping  event  provided  by  the  event  judge 
Andy  Schwartz. 

6.3.  Sub-surface  Clearance  Event  Results 

6.3.1.  Competing  Teams 

The  two  teams  that  competed  in  the  Sub-surface  Clearance  event  were  Team  D4C  and  Team 
UXOD  Automation.  Team  Sky  Research  did  not  participate  in  this  event. 

6.3.2.  Task  Description 

The  sub-surface  clearance  challenge  was  intended  to  demonstrate  the  removal  of  all  seeded 
metallic  items  from  the  supplied  dig  list.  The  seeded  metallic  item  depths  did  not  exceed  1  meter 
and  the  site  was  seeded  with  metallic  items  60mm  in  width  or  greater.  The  dig  list  included  a 
relative  signal  strength  of  the  buried  UXO  items  to  simulate  a  real  geophysical  mapping  product. 
All  seeded  metallic  items  recovered  from  the  clearance  site  were  to  be  placed  in  a  designated 
sub-surface  collection  area  off  of  the  range.  This  was  the  least  automated  task  for  all  the  teams  as 
most  of  the  robotic  operations  were  teleoperated  throughout  the  entire  sub-surface  clearance 
event.  The  operations  for  this  task  were  very  slow  and  inefficient  as  neither  team  instrumented 
their  backhoe  arms  to  know  the  precise  location  where  they  were  digging.  In  addition  their 
technique  of  digging  for  the  items  and  trying  to  visually  see  it  in  their  camera  view  was  very 
poor.  Both  teams  struggled  with  the  subsurface  clearance  and  neither  team  achieved  the 
minimum  score  to  be  award  a  prize  for  this  event. 

During  the  subsurface  clearance  event,  both  competing  teams  had  difficulty  locating  buried  items 
with  sufficient  accuracy  for  extraction.  This  problem  seems  to  stem  from  a  failure  to  properly 
calibrate  differential  GPS  systems.  The  DGPS  systems  used  by  the  competitors  and  the  R2C2 
staff  provide  extremely  high  position  accuracy  relative  to  other  points,  but  the  entire  set  of  points 
may  be  off  in  the  global  reference  frame  unless  post-processing  is  used  to  adjust  the  alignment. 
To  mitigate  any  alignment  problems  the  teams  were  provided  with  marked  localization  points 
they  could  use  to  shift  the  coordinate  plane  of  their  GPS  system  to  the  provided  coordinates. 
Failure  to  properly  shift  coordinate  planes  could  result  in  position  errors  as  great  as  1  meter. 

The  R2C2  staff  demonstrated  the  accuracy  of  the  location  data  provided  to  the  teams  by 
repeatedly  and  reliably  finding  corner  points  and  buried  ISOs  before  and  after  the  competition. 
Furthermore,  the  crew  from  ARA’s  New  England  Division,  who  verified  the  geophysical 
mapping  range  using  a  completely  different  DGPS  system,  was  able  to  do  the  appropriate 
coordinate  shift  with  the  data  provided  to  the  teams.  This  indicates  that  there  was  no  systematic 
problem  with  the  GPS  data  but  care  must  be  taken  when  translating  GPS  data  to  properly  localize 
points. 
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6.3.3.  Team  UXOD  SubSurface  Clearance 

Team  UXOD  Automation  again  used  their  John  Deere  equipment  with  the  skid  steer  employing 
an  equipment  plate-mounted  excavator  bucket  and  the  feller  buncher  modified  to  carry  an 
electromagnet  and  generator  (Figure  53).  Team  UXOD  Automation  used  waypoint  driving  to 
guide  the  skid  steer  to  the  location  of  the  buried  items  and  then  teleoperated  the  system  to  dig  up 
and  look  for  the  buried  UXO.  Once  they  uncovered  an  item  through  visual  verification,  they  used 
waypoint  driving  to  drive  over  the  feller  buncher  and  then  pick  up  the  item  with  the 
electromagnet.  The  feller  buncher  would  sometimes  pick  up  multiple  recovered  UXO  items 
before  using  waypoint  driving  to  take  them  to  the  collection  area. 

Team  UXOD  started  out  very  slow  only  recovering  a  few  items  in  the  first  several  hours  of  the 
event.  They  eventually  changed  their  technique  from  that  of  trying  to  dig  up  the  item  from  a  hole 
where  they  thought  it  was  to  digging  a  trench  and  searching  for  an  item.  This  trenching  method 
proved  to  be  much  more  effective  and  their  retrieval  rate  went  up  significantly.  In  addition,  the 
use  of  the  electromagnet  was  very  successful  in  picking  up  the  items  cleanly  to  take  back  to  the 
collection  area.  Unfortunately,  they  did  not  collect  all  of  the  items  that  were  dug  up.  The  judges 
found  one  UXO  item  left  on  the  surface  next  to  one  of  their  excavated  holes  during  the  scoring  of 
the  range. 

In  addition,  Team  UXOD  had  an  incident  in  this  event  where  they  teleoperated  their  skid  steer 
into  the  hole  they  were  excavating  (Figure  54).  Once  again  the  event  was  halted  and  the  situation 
inspected  for  safety.  They  were  then  allowed  to  recover  their  system  and  continue  the  event. 
Unfortunately,  when  the  system  went  into  the  hole  it  damaged  their  excavator  attachment,  they 
attempted  to  repair  it  unsuccessfully  with  the  excavator  attachment  detaching  shortly  after  they 
resumed  digging.  At  this  time  they  decided  to  switch  to  their  blade  attachment  and  concentrate 
on  backfilling  the  holes  (Figure  55).  Overall,  the  team  recovered  14  of  the  50  buried  items  and 
achieved  a  total  score  of  97.00  out  of  a  possible  250  points  (Figure  56).  Detailed  scoring  sheets 
are  shown  in  Table  . 

In  addition,  Team  UXOD  had  an  incident  in  this  event  where  they  teleoperated  their  skid  steer 
into  the  hole  they  were  excavating  (Figure  54).  Once  again  the  event  was  halted  and  the  situation 
inspected  for  safety.  They  were  then  allowed  to  recover  their  system  and  continue  the  event. 
Unfortunately,  when  the  system  went  into  the  hole  it  damaged  their  excavator  attachment,  they 
attempted  to  repair  it  unsuccessfully  with  the  excavator  attachment  detaching  shortly  after  they 
resumed  digging.  At  this  time  they  decided  to  switch  to  their  blade  attachment  and  concentrate 
on  backfilling  the  holes  (Figure  55).  Overall,  the  team  recovered  14  of  the  50  buried  items  and 
achieved  a  total  score  of  97.00  out  of  a  possible  250  points  (Figure  56).  Detailed  scoring  sheets 
are  attached  in  Section  4.7.5. 
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Figure  53.  Team  UXOD  Sub-surface  Clearance  Task 


Figure  54.  Team  UXOD  Skid  Steer  Hole  Incident 
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Figure  55.  Team  UXOD  Sub-surface  Backfill 


Figure  56.  Team  UXOD  Sub-surface  Recovered  IOI’s 


6.3.4.  Team  D4C  SubSurface  Clearance 

Team  D4C  used  two  robotic  Bobcats  for  this  final  task,  one  with  an  equipment  plate  mounted 
excavator  and  the  other  with  a  custom  made  screener  basket  (Figure  57).  Like  Team  UXOD 
Automation,  Team  D4C  used  waypoint  driving  to  go  to  the  site  of  the  buried  items  and  then 
teleoperation  to  do  the  actual  digging.  Since  they  did  not  have  an  instrumented  backhoe  arm  they 
attempted  to  mark  they  target  location  by  having  the  skid  steer  spin  in  place  over  the  GPS 
coordinate  of  the  suspect  item.  This  left  a  distinctive  circle  of  dirt  from  the  bobcat  tracks  that 
they  then  dug  over.  They  also  were  visually  searching  for  the  items  as  they  dug,  which  proved  to 
be  very  challenging.  They  were  not  finding  many  items  as  they  would  dig  and  then  search  the 
dirt  pile  or  hole  for  an  item  and  then  dig  some  more. 
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Figure  57.  Team  D4C  Sub-surface  Task 


The  second  bobcat  with  the  screener  bucket  was  not  employed  effectively  due  to  technical  issues 
of  operating  the  two  systems  in  close  proximity  to  each  other.  The  original  plan  was  for  them  to 
drop  the  dirt  into  the  screener  bucket  and  then  shake  it  until  an  ISO  item  was  found.  Since  they 
couldn’t  operate  the  two  system  near  each  other  at  the  same  time,  they  had  to  just  visually  search 
for  ISO  in  their  dirt  piles  using  the  on-board  cameras.  As  they  day  wore  on,  D4C  did  not  make 
any  changes  to  their  tactics  and  collected  only  seven  of  the  50  buried  items  (Figure  58).  They 
achieved  a  total  score  of  70.00  out  of  a  possible  250  points.  Detailed  scoring  sheets  are  attached 
in  section  4.7.5. 
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Figure  58.  Team  D4C  Recovered  IOI’s 
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6.3.5.  Scores 


Table  10.  Team  UXOD  Sub-surface  Clearance  Score 


Surface  Clearance  Event  Score 

97  out  a  possible  250  points 

Metric  Categories 

Total  for  each  Metric 

Weight 

System  Task  Performance 

62.00 

50% 

Level  of  Human  Interaction 

102.5 

40% 

Man  Hours  to  Perform  the  Task 

250.00 

10% 

Competition  Level  Metrics 

Points/Violation 

Violations 

Total 

Perimeter  or  No-go  area  violation 

50 

0 

0 

Exclusion  zone  violation 

DQ 

Surface  (>15cm) 

2 

4 

8 

Surface  grade  (>15cm) 

0 

0 

Total  Penalty 

8 

System  Task  Performance  (250)  Points 

62.00 

Possible  Points 

number  of  seeded  items 

number 

collected 

Sub-surface  Seed  Removal 

250 

50 

14 

Level  of  Human  Interaction  (250) 

102.50 

Possible  Points 

(from  Human  Interaction  Judge 

Description  of  Level  of  Human  Interaction 

250 

Near  zero  human  interaction 

100% 

59.0% 

Minimal  interaction 

75% 

Moderate  Interaction 

50% 

Lrequent  Interaction 

25% 

Continuous  Interaction  (completely  tele- 
operated) 

0% 

Man  Hours  to  Perform  the  Task  (250)  Points 

250.00 

Elapsed  Time 

8:00:00 

Number  of  people  on  site 

5 

Total  Man-Hours 

40.00 

Man  Hours  Allotted 

40 
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Table  11.  Team  D4C  Sub-surface  Clearance  Score 


Surface  Clearance  Event  Score 

70  out  a  possible  250  points 

Metric  Categories 

Total  for  each  Metric 

Weight 

System  Task  Performance 

35.00 

50% 

Level  of  Human  Interaction 

68.75 

40% 

Man  Hours  to  Perfomi  the  Task 

250.00 

10% 

Competition  Level  Metrics 

Points/Violation 

Violations 

Total 

Perimeter  or  No-go  area  violation 

50 

0 

0 

Exclusion  zone  violation 

DQ 

Surface  (>15cm) 

0 

0 

0 

Surface  grade  (>15cm) 

0 

0 

Total  Penalty 

0 

System  Task  Performance  (250)  Points 

35.00 

Possible  Points 

number  of  seeded  items 

number 

collected 

Sub-surface  Seed  Removal 

250 

50 

7 

Level  of  Human  Interaction  (250) 

68.75 

Possible  Points 

(from  Human  Interaction  Judge 

Description  of  Level  of  Human  Interaction 

250 

Near  zero  human  interaction 

100% 

73.0% 

Minimal  interaction 

75% 

Moderate  Interaction 

50% 

frequent  Interaction 

25% 

Continuous  Interaction  (completely  tele- 
operated) 

0% 

Man  Hours  to  Perfomi  the  Task  (250)  Points 

250.00 

Elapsed  Time 

8:00:00 

Number  of  people  on  site 

3 

Total  Man-Hours 

24.00 

Man  Hours  Allotted 

24 

6.3. 5.1.  Comments 

This  task  proved  to  be  the  most  difficult  challenge  of  the  competition.  Both  teams,  UXOD  and 
D4C,  experienced  frequent  problems  in  recovering  the  buried  ISO  items.  Predominantly,  these 
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problems  were  associated  with  accurately  geolocating  the  position  of  the  ISO  item  with  respect 
to  their  manipulator  arm  and  dig  tool  coordinate  system  along  with  their  handling  techniques  for 
recovery.  For  example,  both  teams  recovered  ISO  items  in  their  bucket  and  did  not  realize  it  so 
the  inadvertently  buried  the  ISO  into  the  dirt  pile  lying  next  to  the  dig  site.  This  caused  them  to 
overlook  the  item  and  continue  digging,  while  other  times  they  did  not  dig  close  enough  to  the 
exact  spot  of  the  buried  ISO  and  could  not  recover  it. 

6.4.  Overall  Scores 


Table  12.  R2C2  Overall  Scores. 


R2C2  Event  Scores 

Team  UXOD 

Team  D4C 

Team  Sky  Research 

Vegetation  Clearance 

158.62 

177.38 

Did  not  participate 

Surface  Clearance 

161.86 

191.91 

Did  not  participate 

Geophysical  Mapping 

205.70 

165.30 

133.30 

Sub-Surface  Clearance 

97.00 

70.00 

Did  not  participate 

Total  Scores 

623.18 

604.60 

133.30 

6.5.  R2C2  Scoring  Procedures 
6.5.1.  Range  QC  Procedure 

For  the  purposes  of  QC  inspection,  each  range  was  divided  into  grids  using  the  differential  GPS 
system.  After  the  competitors  completed  each  event,  the  comers  of  the  grid  were  marked  with 
red  flags  to  facilitate  inspection  and  each  grid  section  was  manually  inspected  for  any  of  the 
relevant  failures.  Each  grid  section  was  only  counted  as  a  failure  once  for  each  type  of  failure. 

Competitors  were  penalized  if  they  significantly  damaged  the  competition  ranges  by  operating 
their  systems.  The  R2C2  Quality  Control  (QC)  team  checked  for  surface  damage  (either  digging 
ruts,  or  piling  up  material),  grade  changes  (changing  the  entire  surface  of  a  section  of  the  range), 
vegetation  that  was  not  removed  from  the  vegetation  clearance  ranges,  and  tree  stumps  that  were 
not  properly  removed  from  the  vegetation  clearance  ranges.  QC  inspections  were  performed  by 
the  range  judges  supported  by  the  QC  team. 

Tree  stumps  were  not  graded  using  the  grid  system.  Exactly  50  trees  were  left  on  each  vegetation 
clearance  range  at  the  beginning  of  the  competition.  Trees  were  marked  before  the  competition 
and  inspected  afterward  to  determine  if  they  had  been  sufficiently  cut. 
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Figure  59.  Team  D4C  Quality  Control  Results  for  the  Vegetation  Clearance  Range 


R  Grid  On  0  +  Comers  0  0  *  Surf  Damage  [✓]  V  Vegetation  0  o  Stumps  Load  | 

Q  x  Boundary  |  |  Dmg  Quad  Fail  I  I  Veg  Quad  Fail  Drrjjj  ■_  

B  X  5J  |  Help  | 

Figure  60.  Team  UXOD  Quality  Control  Results  for  the  Vegetation  Clearance  Range 

Refer  to  Appendix  E  for  complete  quality  control  results. 

6.5.2.  Surface  Damage  Criteria 

The  failure  criteria  for  Surface  Damage  were  defined  as  follows:  A  pile  of  material  failed  if  it 
was  continuously  above  1 5  cm  in  height  above  the  local  grade  over  a  length  of  at  least  1  meter. 
Holes  constituted  a  failure  if  it  had  a  continuous  depth  of  1 5  cm  below  the  local  grade  over  a 
length  of  at  least  1  meter.  Height  and  Depth  were  measured  with  a  measuring  tape  and  a  long 
rigid  object  (where  necessary).  Piles  were  nonnalized  by  the  QC  team  by  compressing  down  the 
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dirt  to  compact  or  disperse  loose  soil.  Holes  were  not  normalized.  Failures  were  documented 
with  GPS  marking  and  photographs  per  the  documented  procedure  (Figure  61). 


Figure  61.  Surface  Damage  Failures,  (left)  Pile,  (right)  Hole 


6.5.3.  Vegetation  Failure  Criteria 

The  failure  criteria  for  vegetation  residue  was  defined  as:  vegetation  residue  will  fail  if  it  extends 
above  15cm  in  height  above  the  local  grade  over  an  area  of  1  square  meter.  A  theoretical  cylinder 
(of  height  15  cm  and  diameter  lm)  was  proposed  to  define  a  failed  area.  If  a  significant  portion  of 
the  vegetation  inside  the  area  extended  through  the  top  of  the  cylinder,  the  area  constituted  a  grid 
failure  (Figure  62).  This  standard  was  applied  to  uncut  (or  insufficiently  cut)  grass  and  small 
shrubs  inside  the  vegetation  removal  area. 


Figure  62.  Vegetation  Failures 


6.5.4.  Tree  Stump  Failure  Criteria 

The  failure  criteria  for  insufficiently  cut  tree  stumps  was  defined  as:  A  tree  stump  will  fail  if  the 
height  of  the  stump  above  the  local  grade  is  greater  than  8  cm.  The  height  was  measured  on  the 
shortest  side  of  the  stump.  If  the  stump  is  not  cut  evenly  (e.g.  if  it  has  been  mulched),  the 
evaluator  will  stand  on  the  stump  to  compress  any  parts  that  may  be  sticking  up.  The  height  is 
measured  to  the  top  of  the  uncompressible  portion  of  the  stump  (i.e.  the  bottom  of  the  evaluator’s 
shoe)  -  (refer  to  Figure  63  for  an  example). 
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Figure  63.  An  Example  of  Tree  Stump  Failure 


In  all  cases  the  final  decision  for  passing  or  failing  damage,  vegetation,  or  tree  stumps  was 
decided  by  the  range  judge.  The  range  judge  recorded  the  height  of  failed  stumps  and  graded 
each  as  marginal  or  significant  failure. 

6.5.5.  Existing  Damage  Mitigation 

All  ranges  were  checked  prior  to  the  competition  and  both  the  subsurface  and  vegetation 
clearance  ranges  were  deemed  to  have  existing  damage  that  would  count  as  a  failure.  To  ensure 
that  no  team  was  penalized  for  existing  damage,  the  fields  were  either  normalized  or  the  damage 
was  documented. 

The  vegetation  clearance  ranges  were  normalized  by  the  QC  team.  Using  shovels  and  rakes, 
mounds  that  came  close  to  the  15  cm  limit  were  spread  out  and  holes  that  came  close  to  the  15  cm 
depth  limit  were  filled  in.  The  existing  damage  in  the  Vegetation  Clearance  ranges  was  minimal 
and  easy  to  mitigate. 

The  vegetation  ranges  also  had  several  tree  stumps  that  exceeded  the  8  cm  limit.  Existing  stumps 
were  either  cut  down  to  a  passing  state  or  marked  by  cutting  an  X  into  the  top  of  the  stump.  To 
further  mitigate  this  concern,  the  50  trees  that  were  targets  for  the  competitors  were  marked  with 
white  paint  at  the  base  of  the  tree  trunk.  Failing  trees  that  were  left  by  the  competitors  had 
remnants  of  the  white  paint  on  their  trunks.  These  techniques  greatly  simplified  identification  of 
stump  failures. 

The  subsurface  clearance  ranges  both  had  some  existing  surface  damage  (including  a  road 
through  each  field  and  deep  tire  tracks.  It  was  deemed  infeasible  to  flatten  out  the  entire  surface 
of  the  ranges.  Instead  the  QC  team  checked  each  range  for  damage  before  the  competition.  All 
damage  that  could  potentially  cause  a  failure  was  GPS  tagged  and  photographed.  In  all  cases,  the 
existing  damage  was  easily  differentiated  from  new  damage,  and  no  competitor  caused  surface 
damage  failures  that  coincided  with  existing  damage. 
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6.5.6.  Range  QC  Summary  Rulings 

Based  on  the  rulings  of  the  range  judges,  the  following  QC  checks  were  deemed  unnecessary  and 
were  therefore  not  performed.  All  other  QC  checks  were  perfonned  per  the  procedure.  This 
refers  specifically  to  the  QC  procedures  performed  by  the  QC  team  as  outlined  in  the  GPS  field 
manual  and  does  not  include  other  assessments  of  competitor  perfonnance. 

•  Surface  Clearance  Ranges:  Surface  Damage  and  Grade  Change  checks  were  deemed 
unnecessary  as  there  was  no  apparent  damage  to  either  range. 

•  Geophysical  Mapping  Range:  Surface  Damage  and  Grade  Change  checks  were  deemed 
unnecessary,  but  the  QC  team  was  asked  to  verify  the  position  of  the  obstacles  after  a 
competitor  robot  impacted  one  of  them.  No  movement  of  the  obstacle  was  detected. 

•  Vegetation  Clearance  Range:  Grade  Change  QC  was  deemed  unnecessary 

•  Subsurface  Clearance  Range:  Grade  Change  QC  was  deemed  unnecessary 

6.5.7.  Human  Interaction  Monitoring 

During  each  event,  the  level  of  human  interaction  was  monitored  for  each  team/system.  R2C2 
staff  members  were  selected  and  trained  as  Human  Interaction  (HI)  monitors.  Two  observers 
were  selected  as  primaries  for  each  event  and  the  same  observers  monitored  both  teams  for  a 
given  event  to  ensure  consistency  in  record  keeping.  Alternate  HI  monitors  and  the  HI  judge  did 
take  over  for  brief  periods  to  relieve  the  primary  observers  as  necessary. 

HI  monitors  recorded  all  interactions  on  printed  sheets.  The  first  line  of  the  first  sheet  was  open 
for  notes  about  the  competitor’s  control  setup.  These  notes  listed  the  number  of  robots,  number 
or  control  units,  and  number  of  operators.  The  abbreviations  were  defined  for  the  robots  and 
computers  used.  After  the  initial  notes,  the  record  sheets  had  lines  with  start  and  stop  times,  a  list 
of  operational  modes  (e.g.  waypoint  following,  teleoperated,  paused,  etc.),  and  a  space  for  notes. 
HI  monitors  noted  the  time  of  any  interactions,  and  made  notes  any  time  an  operator  interacted 
with  the  competitor  systems. 

Human  Interactions  were  defined  as  any  time  a  user  entered  a  command  that  changed  the  state  of 
the  system.  This  included  driving  a  robot,  initiating  or  changing  a  waypoint  maneuver,  moving 
actuators,  moving  cameras,  inputting  waypoints,  stopping  the  robots,  doing  maintenance, 
manually  driving  a  robot,  etc.  Pure  monitoring  activities  (e.g.  observing  video  feeds,  or 
downloading  and  graphing  Geophysical  data)  were  not  counted  as  interaction  even  if  the  user 
was  using  a  computer  to  view  the  data. 

6.5.8.  Video  boundary  monitoring 

Each  range  had  cameras  placed  to  monitor  the  boundaries  in  order  to  catch  boundary  violations 
(Figure  64).  Three  or  more  cameras  were  used,  at  least  one  pointed  down  each  of  the  three  sides 
on  which  a  boundary  violation  could  occur.  The  range  entrance/exit  edge  was  not  monitored  and 
competitors  were  not  penalized  for  crossing  this  edge. 
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Figure  64.  Minimal  Camera  Setup  Diagram  (3  Cameras) 


According  to  the  R2C2  rules,  the  teams  would  be  penalized  if  their  robot  went  more  than  3m 
beyond  the  30m  exclusion  zone.  The  boundary  cameras  were  set  up  on  the  comers  of  the  33m 
boundary  rectangle  and  pointed  along  the  edges  at  a  post  on  the  opposite  corner.  If  a  vehicle 
went  out-of-bounds  the  post  would  be  blocked  and  a  boundary  violation  was  known  to  have 
occurred.  All  cameras  were  wired  back  to  a  common  control  point  for  monitoring  and  recording. 
All  boundary  video  was  recorded  and  archived,  and  as  violations  occurred  a  line  judge  noted  the 
violation  in  their  logbook. 

Although  a  minimum  of  three  cameras  were  needed  for  each  range,  some  ranges  needed 
additional  cameras  and  posts  because  of  rough  topography  that  did  not  allow  line  of  sight  for  the 
full  distance  of  an  edge.  Each  vegetation  range,  for  instance,  needed  eight  cameras  to  monitor  the 
entire  range  boundary. 

6.5.9.  Human  Interaction  Logging 

As  part  of  a  competitor’s  autonomy  assessment,  the  Autonomy  Judge  instituted  a  method  for 
documenting  human  interactions  referred  to  as  Human  Interaction  Logs.  During  each 
competition  one  or  more  representative(s)  from  the  Human  Interaction  Team  would  record  all 
interactions  instance  with  competitor’s  robotic  platforms  in  a  method  dictated  by  the  Human 
Interaction  Judge.  A  Human  Interaction  Instance  was  defined  as  any  time  a  user  entered  a 
command  that  changed  the  state  of  the  system.  This  included  driving  a  robot,  initiating  or 
changing  a  waypoint  maneuver,  moving  actuators,  moving  cameras,  inputting  waypoints, 
stopping  the  robots,  doing  maintenance,  manually  driving  a  robot,  etc.  Pure  monitoring  activities 
(e.g.  observing  video  feeds,  or  downloading  and  graphing  Geophysical  data)  were  not  counted  as 
interaction  even  if  the  user  was  using  a  computer  to  view  the  data. 

To  further  maintain  consistency,  beyond  just  common  definitions,  each  event  had  a  Primary 
Human  Interaction  Team  Member  (PHITM)  and  an  Assistant  Human  Interaction  Team  Member 
(AHITM)  both  of  which  were  dedicated  to  that  event  class  in  an  effort  to  alleviate  potential 
subtle  style  differences.  An  exception  was  made  for  Geophysical  Mapping,  which  was  judged  as 
needing  only  a  PHITM.  Each  Human  Interaction  Team  member  focused  on  specific  Human 
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Machine  Interfaces  (HMIs)  discovered  during  a  pre-competition  interview  with  each  team.  The 
AHITM  was  responsible  for  setting  up  and  monitoring  a  video  log  of  the  event  which  was  placed 
to  record  as  much  human  interaction  as  possible  without  being  a  hindrance  to  the  competitors 
(This  video  was  referred  to  as  the  Human  Interaction  Video  Log).  AHITMs  and  PHITMs  also 
acted  as  relief  for  each  other  such  that  there  was  never  a  gap  in  observations.  Before  the  event 
the  PHITM  created  a  naming  convention  for  each  HMI  that  was  used  throughout  the  specific 
event.  The  naming  convention  along  with  general  competitor  strategy  notes  were  documented  on 
the  back  of  the  first  page  in  Human  Interaction  Logs. 

Human  Interaction  Logs  were  composed  of  preformatted  templates  that  described  specific 
interaction  instances.  The  first  page  of  a  Human  Interaction  Log  (top  lines  shown  in  Figure  65) 
gave  the  Human  Interaction  Team  a  place  to  insert  basic  notes  including  number  of  robots  and 
number  of  HMI’s.  Each  interaction  instance  was  appended  with  a  human  interaction  instance 
number.  Human  Interaction  Instances  were  generally  classified  into  two  types:  instantaneous  and 
on-going.  Instantaneous  interactions  were  characterized  as  taking  less  than  30  seconds  to 
accomplish  and  were  indicated  by  only  listing  a  start  time.  On-going  interactions  took  more  than 
30  seconds  and  had  both  a  start  and  end  time.  (NOTE:  Times  were  recorded  by  Synchronized 
Atomic  clocks  given  to  the  Human  Interaction  Team). 


1  1  II  1  1  1  I 

11 1 1 1 1 1 

SB#  Robotic  Ranse 

Clearance  Competition  Human  Interaction  Log 

Starting  Notes: 

Types  of  interfaces  and  what  they  do: 

Number  of  operators  and  robots: 

1 

Start: 

End: 

Description: 

TO  WPT  VC  SA  Anomaly  Pause 

Comments 

Figure  65.  Human  Interaction  Log  Form 


Additionally,  interactions  were  sub-classified  by  circling  one  of  the  description  identifiers  of  the 
Human  Interaction  Instance  as  described  in  Table  13.  Further,  a  comments  section  was  provided 
to  the  Human  Interaction  Team  to  write  more  details  (when  merited)  about  the  interaction. 

After  every  competition  the  Human  Interaction  Log  was  hand  delivered  to  the  Autonomy  Judge 
after  a  tabulation  of  interaction  classification  (instantaneous  or  on-going)  as  well  as  a  summation 
of  on-going  type  human  interaction  minutes.  The  above  mentioned  Human  Interaction  Video 
Log  was  also  made  available  to  the  Autonomy  Judge  during  judging  sessions. 
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Table  13.  Human  Interaction  Identifier  Definitions 


Identifier 

Definition 

TO 

Tele-operation;  A  Human  Machine  interaction  at  directly  controlled 
movement 

WPT 

Way  Point  Tracking;  A  automated  method  for  vehicle  navigation 

VC 

Vector  Controlled;  A  partially  automated  method  for  dictating  control  by  the 
system  state 

SA 

Scripted  Action:  A  single  Human  Interaction  which  cause  the  machine  to  run 
a  script 

Anomaly 

Anomaly;  Anything  not  covered  by  other  Identifiers 

Pause 

Pause;  An  operational  pause  in  the  event 

6.6.  Competition  Outcomes 

The  competition  resulted  in  monetary  awards  to  two  different  teams  and  the  crowning  of  an 
overall  winner.  However,  it  is  interesting  to  note  that  the  overall  winner  only  won  a  single  event, 
but  was  the  only  team  to  qualify  for  the  overall  award  by  achieving  at  least  a  minimum  score  in 
each  event. 

6.6.1.  Other  Observations 

Despite  the  incentives  for  increasing  autonomy,  both  of  the  teams  who  competed  in  all  the  events 
relied  on  a  high  level  of  tele-operation  for  events  which  required  dynamic  and  rapid  decision 
making  and  flexibility  and  uncertainty  in  planning  (such  as  the  vegetation  clearance  and 
subsurface  clearance  events).  Conversely,  when  the  events  were  amenable  to  simple  path 
planning  algorithms  (such  as  the  geophysical  mapping  and  surface  clearance  events),  teams  used 
more  autonomy  in  the  form  of  waypoint  following  applications. 

6.6.2.  Category  Winners  and  Overall  Winner 

“The  results  of  the  R2C2  were  announced  17  August  at  the  Association  of  Unmanned  Vehicle 
Systems  International  (AUVSI)  North  America  2011  conference  in  Washington,  D.C.  Team  D4C 
won  both  the  Vegetation  Clearance  and  Surface  Clearance  events  and  was  awarded  $500,000. 
Team  UXOD  -  won  the  Geophysical  Mapping  event  and  received  and  additional  $1,000,000 for 
achieving  the  highest  overall  total  competition  score.  Their  total  award  was  $1,250,000.  ” 

These  were  the  words  posted  to  the  competition  website  after  the  winners  of  the  competition 
were  announced  at  AUVSI  North  America  201 1.  A  few  weeks  later,  an  official  press  release 
from  OSD  was  posted  to  the  competition  website  to  officially  announce  the  results.  The  text  of 
that  press  release  appears  here. 

“The  Robotic  Range  Clearance  Competition  (R2C2)  Team;  the  Office  of  the  Secretary  of 
Defense  (OSD),  Land  Warfare  and  Munitions ;  the  Air  Force  Research  Laboratory;  and  the  U.S. 
Army  Corps  of  Engineers  would  like  to  announce  the  winners  from  the  Robotic  Range  Clearance 
Competition  (R2C2)  held  August  7-14,  2011  at  Camp  Guernsey,  WY. 
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The  Office  of  Secretary  of  Defense  (OSD)  in  collaboration  with  the  US  Air  Force  and  US  Army 
conducted  a  Robotic  Range  Clearance  Competition  (R2C2)  to  foster  the  ability  to  clear  training 
ranges  of  debris  and  unexploded  ordnance  (UXO)  using  robotic  technologies.  The  purpose  of  the 
Robotic  Range  Clearance  Competition  (R2C2)  was  to  quickly  tap  into  the  innovation  and 
ingenuity  of  the  commercial  robotic  technology  sector  to  improve  the  safety  and  effectiveness  of 
the  four  tasks  traditionally  associated  with  range  clearing  operations:  1)  Vegetation  Clearance, 
2)  Surface  Clearance,  3)  Geophysical  Mapping,  4)  Subsurface  Clearance. 

The  R2C2  Competition  teams  were: 

1.  Team  UXOD  Automation  -  comprised  of  Kairos  Autonomi,  Autonomous  Solutions,  SAIC, 
Zonge  Engineering,  Vallon,  WM  Robots,  VKR,  Inc.,  and  John  Deere 

2.  Team  D4C  -  comprised  of  ECC,  QinetiQ  North  America,  and  Bobcat 

3.  Team  Sky  Research 

The  R2C2  Competition  Directors  would  like  to  congratulate  the  competing  teams  and  thank  the 
R2C2  judges  and  support  staff  for  making  the  R2C2  a  successful  and  rewarding  event.  ” 
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7.  CONCLUSIONS 


The  Robotic  Range  Clearance  Competition  successfully  met  its  primary  objective  of 
incentivizing  industry  to  develop  robust,  fieldable  solutions  to  address  the  problem  of  munitions 
range  clearance.  Traditional  range  clearance  operations  are  manpower  intensive,  time 
consuming,  dangerous  and  expensive.  Data  from  AFRL  and  the  Anny  Corps  robotic  range 
clearance  technology  experiments  showed  the  strong  potential  for  significant  reductions  in  the 
time  and  cost  required  to  conduct  range-clearing  operations  —  the  possibility  of  reduced  time  by 
two-thirds  and  cost  by  one-third.  However,  there  was  no  automated  “commercial  off  the  shelf’ 
solutions  available  for  procurement  at  the  time  of  the  competitions  initiation. 

The  prize  competition  mechanism  was  selected  to  quickly  tap  into  the  innovation  and  ingenuity 
of  the  commercial  robotic  technology  sector  to  meet  this  need.  The  Department  of  Defense 
intended  for  the  R2C2  to  result  in  viable  systems  that  can  be  procured  and  placed  into  service 
clearing  ranges  more  efficiently  than  the  manpower  intensive  methods  historically  employed. 

The  results  of  the  competition  prove  that  industry  is  ready  to  respond  to  these  needs  with  robotic 
systems  capable  of  performing  the  necessary  range  clearance  tasks.  The  unmanned  systems  used 
for  the  competiton  demonstrated  the  ability  to  perform  all  the  tasks  successfully  in  a  reasonable 
time  frame  and  in  an  operational  environment.  The  systems  were  operated  almost  continually  for 
two  weeks  with  relatively  few  failures  of  the  machines  or  unmanned  system  components.  The 
only  task  that  was  not  awarded  a  prize  (subsurface  clearance)  was  due  to  poor  technique 
selection  and  lack  of  proper  instrumentation  by  the  teams  rather  than  inability  of  the  equipment 
to  achieve  the  task. 

Despite  the  competition  incentives  for  increasing  autonomy,  the  teams  relied  on  a  high  level  of 
tele-operation  for  events  which  required  dynamic  decision  making  and  flexibility  to  deal  with 
uncertainty  in  planning  (such  as  the  vegetation  clearance  and  subsurface  clearance  events). 
Conversely  for  the  simpliler  tasks  (such  as  the  geophysical  mapping  and  surface  clearance 
events),  teams  used  more  autonomy  in  the  fonn  of  path  planning  &  waypoint  following 
algorithms. 

The  US  Army  Corp  of  Engineers  is  currently  planning  and  preparing  to  release  a  request  for 
proposal  for  Robotic  Range  Clearance  Services  based  on  the  results  of  the  competition.  The 
ultimate  aim  of  this  effort  and  future  contract  is  to  efficiently  return  to  productive  use  the 
millions  of  acres  currently  encumbered  with  spent  training  rounds  and  debris. 
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8.  RECOMMENDATIONS 


This  report  documented  the  scope,  preparation,  and  execution  of  the  R2C2  sponsored  by  the 
JGRE,  the  USACE,  and  AFRL.  The  overall  competition  was  highly  succesful  and  met  the 
primary  objectives  of  the  sponsors,  but  there  are  a  few  recommendations  that  will  improve  any 
subsequent  DoD  prize  competitions. 

The  competition  event(s)  should  be  held  near  a  large  metropolitan  area  or  military  installation. 
The  remote  nature  of  the  Camp  Guernsey  site  made  it  a  challenge  to  support  the  competition  and 
encourage  relevant  stakeholders  to  participate  and  attend. 

Competitors  should  be  incentivized  to  compete  with  seed  funds  to  ensure  an  adequate  number 
and  viable  field  of  competitors.  Many  viable  competitors  with  promising  solutions  were  intially 
attracted  to  participate  but  were  unable  to  secure  funding  and  had  to  drop  out. 
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Robotic  Range  Clearance  Competition 

August  7-1 5,  201 1 ,  Camp  Guernsey,  WY 


Introduction 

The  Office  of  Secretary  of  Defense  (OSD)  in 
collaboration  with  the  US  Air  Force  and  US  Army 
is  conducting  a  Robotic  Range  Clearance 
Competition  (R2C2)  to  foster  the  ability  to  clear 
training  ranges  of  debris  and  unexploded 
ordnance  (UXO)  using  robotic  technologies.  The 
purpose  of  R2C2  is  to  quickly  tap  into  the 
innovation  and  ingenuity  of  the  commercial 
robotic  technology  sector  to  improve  the  safety 
and  effectiveness  of  the  four  tasks  traditionally 
associated  with  range  clearing  operations:  1) 
Vegetation  removal,  2)  Surface  clearance,  3) 
Geophysical  mapping,  and  4)  Subsurface 
clearance. 

Experience  to  date  with  robotic  range 
clearance  suggests  a  significant  potential  for 
time  and  cost  savings  as  well  as  increases  in 
the  safety  for  the  personnel 
conducting  these  activities.  The  ultimate  aim  is  to 
return  to  productive  use  the  millions  of 
acres  currently  encumbered  with  spent 
training  rounds  and  debris.  The  Department  of 
Defense  (DoD)  intends  for  this  Competition  to 
result  in  viable  systems  that  can  be  procured 
and  placed  into  service  clearing  ranges. 


Background 

Range  clearance  operations  as  currently 
conducted  are  manpower  intensive,  time 
consuming,  dangerous,  and  expensive.  Initial 
data  from  robotic  range  clearance  technology 
development  efforts  indicate  the  strong  potential 
for  significant  reductions  in  the  time  and  cost 
required  to  conduct  range  clearing  operations. 
Experiments  to  date  indicate  the  possibility  of 
reducing  range  clearance  times  by  two  thirds 
and  costs  by  one  third  if  automated  clearing 
equipment  is  used.  Currently  there  are  no 
automated  “commercial  off  the  shelf”  solutions 
available  to  the  Department  to  procure.  The 
traditional  approach  of  establishing  an  R&D 
program  that  can  be  transitioned  into  a 
development  and  acquisition  program  is  not 
possible  within  the  desired  quick  turn-  around 
timeframe  as  none  of  the  uniformed  services 
have  programmed  funding  for  this  traditional 
approach.  OSD  AT&L  is  exercising  the  statutory 
authority  under  Section  2374a  of  title  10  United 
States  Code  as  amended  by  Section  212  of  the 
John  Warner  National  Defense  Authorization  Act 
for  Fiscal  Year  2007  Public  Law  109-364  to  offer 
a  cash  prize  for  the  development  of  robotic 
range  clearing  applications.  The  goal  is  to 
provide  the  desired  range  clearance  capabilities 
in  a  significantly  shorter  amount  of  time. 
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Clearance  Competition 


I  j  i  1 1 

BgMI  Robotic  Ranse 


Scope 

The  tasks  associated  with  range  clearance 
tnat  MKeiy  nave  tne  greatest  potential  tor 
applying  ground  robotics  technology  include 
automated  vegetation  clearance,  automated 
surface  debris  clearance,  automated 
geupnysicai  mapping,  anu  auiumaieu 
subsurface  anomaly  excavation.  The 
competition  will  assess  the  ability  of 
competitor  systems  to  provide  increased 
safety  and  operational  effectiveness  to 
range  clearance  operations.  The  competitor 
systems  are  expected  to  have  applied 
robotics  technology  to  all  or  some 
appropriate  combination  of  the  inherent 
range  clearance  tasks  in  a  range  clearance 
operation.  Because  the  competition  is 
focused  on  increasing  safety  and 
operational  effectiveness  via  robotics 
automation  as  well  as  reducing  time  and 
cost,  competitors  are  not  expected  to 
attempt  to  develop  improved  vegetation 
removal  tools  or  geophysical  detection  and 
identification  sensor  technology. 

Schedule 

The  competition  will  be  conducted  from 
August  7-15,  2011  with  a  Media/Visitor  day 
scheduled  for  Thursday  August  1 1 .  R2C2 
competitor  practice  and  setup  activities  will 
be  conducted  August  1-5.  Each  competition 
event  will  be  conducted  over  an  8  hour  shift 
to  represent  1  work  day  starting  at  8am  until 
5pm  local  time.  Depending  on  the  day,  up  to 
two  teams  will  be  competing  simultaneously 
in  separate  events. 

Events  in  the  schedule  are  abbreviated  as  follows: 
Geophysical  Mapping  (MAP),  Vegetation  Clearance 
(VEG),  Surface  Clearance  (SURF),  and  Sub-surface 
Clearance  (SUB). 


R2C2  Competition  Schedule 


Day/Team 

Team  1 

Team  2 

Team  3 

Aug  7 

SUN 

Safety  Test 

Safety  Test 

Safety  Test 

Aug  8 

MON 

MAP 

VEG 

Aug  9 

TUE 

MAP 

Aug  10 

WED 

VEG 

MAP 

Aug  11 

THU 

Media  and  Visitor  1 

ay 

Aug  12 

FRI 

SURF 

SUB 

Aug  13 

SAT 

SUB 

SURF 

Aug  14 
Aug  15 

SUN 

MON 

Weather  Days 

Location 


The  competition  will  be  held  at  Camp 
Guernsey,  WY,  located  in  eastern  Wyoming 
approximately  100  miles  north  of  Cheyenne. 
Camp  Guernsey  is  an  Army  National  Guard 
Facility  with  a  large  Air  Force  presence 
consisting  of  over  77,000  acres  of 
maneuver  area,  training  ranges,  and  state- 
of-the-art  facilities.  A  dedicated,  fully 
qualified  staff  currently  supports  over 
240,000  customer  man-days  annually  of 
combat  training  24/7/365. 
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Notional  R2C2  Competition  Site  Layout. 


Site/Event  Description 

The  R2C2  events  will  be  conducted  on  the 
northern  training  areas  at  Camp  Guernsey, 
WY,  approximately  25  miles  from  the  main 
camp.  The  surrounding  area  is  comprised  of 
mixed  grass  and  sagebrush  upland  plains 
with  sections  of  lodgepole  pines  and 
cottonwoods.  These  competition  sites  are 
all  contained  within  a  600-acre  continuous 


area  at  an  elevation  of  approximately  5000 
ft.  The  sites  are  representative  of  military 
impact  areas  and  training  ranges  that  are  in 
need  of  unexploded  ordnance  (UXO) 
clearance.  Note:  this  area  has  no  known 
history  of  UXO. 
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Lodging  and  Travel  Information 

Lodging  for  the  teams  and  visitors  is  being 
provided  on  Camp  Guernsey  in  billeting. 
Appropriate  accommodations  are  available 
for  all  ranks  and  grades  of  personnel. 
Transportation  will  be  provided  from  Camp 
Guernsey  to  the  event  and  back  each  day. 
Meals  are  being  provided  by  the  camp  mess 
and  will  be  available  at  the  competition  area 
and  in  Camp  Guernsey. 


The  closest  airport  with  airline  service  is 
Cheyenne  Regional  Airport  (CYS), 
Cheyenne,  WY.  Rental  cars  are  available.  It 
is  approximately  a  1.5  hour  drive  to  Camp 
Guernsey  driving  I-25N  and  taking  exit  92 
onto  US-26  E  toward  Guernsey/Torrington. 
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Points  of  Contact 


Air  Force  Research  Laboratory  (AFRL1 


Mr.  Brian  Skibba 

Robotic  Range  Clearance  Competition  Director 
AFRL/RXQE  Robotics  Research  Team 
(850)  283-3725 
brian.skibba@tyndall.af.mil 


Joint  Ground  Robotics  Enterprise  (JGRE1 

Mr.  Robert  Maline 
Director,  JGRE  OUSD 
(AT&L)/PSA,  LW&M 
(703)  693-9414 
robert.maline@osd.mil 

U.S.  Army  Corps  of  Engineers  MJSACE1 

Mr.  Plyler  McManus 

Chief,  Ordnance  and  Explosives  Design  Center 
US  Army  Engineering  and  Support  Center,  Huntsville 
(256)  895-1709 

samuel.p.mcmanus@usace.army.mil 


Camp  Guernsey.  Wyoming 

David  W.  Herder,  LTC,  WY  ANG 
Deputy  Garrison  Commander 
(307)  836-7730 
david.herder@us.army.mil 


Joint  Training  and  Experimentation  Center  (JTEC1 

Mr.  David  A.  Malek 
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Introduction 


Purpose 

The  purpose  of  this  document  is  to  record  the  results  of  the  R2C2  validation  trials  held  at 
Camp  Guernsey,  WY  during  the  week  of  18-22  October  2010  and  to  make 
recommendations  for  any  changes  to  the  existing  competition  rules  or  procedures. 


Scope 

This  document  will  describe  the  validation  trials,  how  they  were  conducted,  an  overview 
of  the  results,  and  recommended  changes  to  rules  and  procedures. 


Objectives 

The  objectives  of  the  trials  held  at  Camp  Guernsey,  WY  during  the  week  of  18-22 
October  2010  were  to  validate  the  scoring  criteria  for  the  R2C2  competition,  to  establish 
baselines  for  the  scoring  components  (e.g.,  expected  time  and  manpower  requirements), 
and  to  report  the  results  of  the  trials  back  to  the  R2C2  Oversight  IPT  to  serve  as  an 
empirical  basis  on  changing  scoring  criteria  and  competition  procedures,  if  necessary. 


Validation  Trials 

The  validation  trials  were  held  at  Camp  Guernsey’s  North  Training  Area,  located  north  of 
Guernsey,  WY,  shown  in  Figure  1.  The  purpose  of  these  trials  was  to  act  as  a  dry  run  for 
the  actual  competition,  scheduled  to  be  held  in  2011,  and  as  a  way  to  validate  the 
competition  scoring. 
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Figure  1.  Map  of  Wyoming  Showing  General  Location  of  R2C2  Ranges. 
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ARA’s  JTEC  group  organized  the  logistics,  prepared  the  ranges,  integrated  the 
automation  hardware  with  the  ARTS  vehicles,  instrumented  the  range  boundaries  with 
video,  provided  the  system  tele-operator,  and  supplied  the  human-machine  interaction 
referee  for  the  demonstration.  The  US  Army  Corps  of  Engineers’  Andy  Schwartz 
oversaw  geophysical  mapping  tasks.  Mr.  Brian  Skibba  of  AFRL/RXQF  and  Mr.  Randy 
Williams  of  JGRE  represented  the  R2C2  Oversight  IPT  and  observed  the  trials.  ARA’s 
Northeast  Division  supported  the  baseline  and  demonstration  geophysical  mapping  by 
providing  their  Army  Night  Vision  Labs  sponsored  UXO  Trailer  and  the  technical 
expertise  to  operate  the  sensors  and  data  collection  software.  The  University  of  Florida’s 
Cl  MAR  lab  teamed  with  ARA  to  provide  the  hardware  and  software  to  automate  the 
range  clearance  machinery.  Attendees  are  shown  in  Figure  2  with  an  autonomous 
ARTS,  the  UXO  Trailer,  and  a  robotic  Gyro-Trac  mulching  machine. 
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The  Ranges  and  Pit  Area  for  the  Validation  Trials 

The  ranges  and  the  pit  area  for  the  validation  trials  were  located  in  the  North  Training 
Area  of  Camp  Guernsey,  WY.  Figure  3  shows  a  Google  Earth  image  of  all  ranges  that 
were  constructed  and  the  location  of  the  pit  area. 


Figure  3.  Locations  of  R2C2  Ranges  Being  Constructed  in  Camp  Guernsey  NTA. 


Ranges 

Seven  ranges  were  created  to  simulate  R2C2  competition  ranges.  One  geophysical 
mapping  range  was  created,  along  with  two  surface  clearance  ranges,  two  Sub-surface 
clearance  ranges,  and  two  vegetation  clearance  ranges.  A  pit  area  was  also  created  to 
house  a  command  trailer,  a  mess  tent,  a  maintenance  tent,  generators,  and  outdoor 
lighting  rigs.  A  picture  of  the  pit  area  is  shown  in  Figure  4. 
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Figure  4.  The  Pit  Area  for  the  R2C2  Validation  Trials. 

Range  Preparation  Process 

The  R2C2  logistics  team  prepared  the  ranges  by  surveying  in  the  corner  posts  for  each 
range.  Then  each  range  was  surveyed  and  seeded  according  to  the  range  seeding  plan. 
The  geophysical  mapping  range  seeding  work  plan  is  included  as  Appendix  A. 

The  logistics  team  installed  video  monitoring  equipment  for  the  back  boundary  of  all 
ranges  and  adjusted  camera  locations  to  also  monitor  side  boundaries  of  each  range 
when  it  was  active.  Video  monitoring  allowed  the  team  to  establish  when  vehicles 
exceeded  range  boundaries,  resulting  in  a  boundary  penalty.  Video  monitoring  is 
described  in  more  detail  later  in  this  document. 

Geophysical  Mapping  Range 

After  boundary  surveying,  the  geophysical  mapping  range  was  mowed  and  cleared  of  all 
vegetation  taller  than  8  cm.  The  logistics  team  then  placed  railroad  tie  obstacles  on  the 
range  in  random  locations.  A  picture  of  the  range  is  shown  in  Figure  5. 


Figure  5.  Geophysical  Mapping  Range  with  Railroad  Tie  Obstacles. 

The  logistics  team  then  performed  a  baseline  geophysical  survey  of  the  range  using  a 
UXO  trailer  built  by  ARA’s  New  England  Division.  The  NED  UXO  Trailer  is  a  towed  trailer 
featuring  a  Minelab  Single  Transmit  Multiple  Receive  (STMR)  metal  detection  array.  The 
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detection  array  is  a  Time  domain  electromagnetic  induction  sensor.  The  trailer  is  shown 
in  Figure  6. 


Figure  6.  The  ARA/NED  UXO  Trailer  Towed  by  the  ARTS. 


After  the  baseline  was  complete,  the  data  were  examined  to  plan  the  optimal  locations 
for  the  Industry  Standard  Object  (ISO)  seeds.  ISO  seed  locations  were  chosen  to 
correspond  to  areas  having  lower  baseline  noise  levels. 

We  then  seeded  the  range  with  126  small  and  medium  ISOs  (63  of  each).  The  team, 
with  assistance  from  Andy  Schwartz  of  USACE,  dug  the  seed  locations,  placed  the 
ISOs,  surveyed  and  recorded  their  locations  in  latitude,  longitude,  and  depth  in 
centimeters,  and  buried  the  ISOs  according  to  the  procedures  outlined  in  the  seeding 
plan.  Localization  was  performed  with  an  Ashtech  Magellan  ProMark  3  RTK  GPS  survey 
kit  shown  in  Figure  7. 


Figure  7.  Ashtech  Magellan  ProMark  3  RTK  Surveying  Equipment  in  Use. 
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Seeding  Procedures 

Using  the  analyzed  baseline  data,  the  logistics  team  determined,  with  recommendations 
by  the  project  geophysicist,  the  approximate  locations  of  each  of  the  63  medium  and  63 
small  ISO  being  used  as  seeds. 

All  seed  locations  were  selected  so  that  no  other  anomaly,  either  pre-existing  or  another 
planned  seed,  was  within  2  meters  of  a  seed  location.  All  seed  items  were  either  the 
small  or  medium  ISO.  (Only  localized  sensor  responses  were  needed  to  assess  how 
well  robotic  systems  measure  where  EM  data  are  collected.  Therefore,  large  ISOs  were 
not  needed.)  Examples  of  these  ISOs  are  shown  in  Figure  8. 


Figure  8.  Picture  of  ISOs  Required  for  the  R2C2. 
Size  descriptions  for  each  of  the  ISOs  are  presented  in  Table  1. 

Table  1.  ISO  Descriptions  and  Specifications. 


Item 

Nominal 
Pipe  Size 

Outside 

Diameter 

Length 

Part 

Number1 

ASTM 

Specification 

Small  ISO 

1" 

1.315" 

(33  nun) 

4" 

(102  nun) 

44615K466 

A53/A773 

Medium  ISO 

2" 

2.375" 

(60  mm) 

8" 

(204  nun) 

44615K529 

A53/A773 

Large  ISO 

4” 

4.500" 

(115  mm) 

12" 

(306  nun) 

44615K137 

A53/A773 

1  Part  number  from  the  McMaster-Carr  catalog. 


All  seeds  were  buried  perpendicular  to  the  local  ground  surface  (i.e.,  vertical  with  respect 
to  the  local  topography).  Coordinate  accuracy  was  +/-2cm  in  x,  y,  and  z  vertical 
orientation  will  be  accurate  to  +/-5  degrees. 

All  seeds  were  placed  in  the  ground  at  a  depth  of  either  5  or  7  times  their  diameter.  All 
depth  measurements  were  to  the  center  of  the  items.  This  required  that  for  each  size  of 
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ISO,  an  offset  representing  the  distance  from  the  end  of  the  ISO  (the  top,  when  buried 
vertically)  to  its  center,  was  added  to  each  survey  measurement  if  that  measurement 
was  taken  at  the  top  of  each  ISO.  Figure  9  shows  a  small  ISO  being  surveyed  prior  to 
burial. 


Figure  9.  An  ISO  Being  Surveyed  in  its  Burial  Location. 


Since  obstacles  were  placed  on  the  range,  no  ISOs  were  seeded  within  the  buffer 
surrounding  these  obstacles  (approximately  1  meter). 

The  burial  team  ensured  that  the  ISOs  were  buried  at  the  appropriate  depth  by  double 
checking  that  the  ISO  for  burial  was  the  appropriate  size  and  that  the  depth  was  5  to  7 
times  the  diameter  of  the  ISO.  The  burial  team  surveyed  each  burial  location  (x,  y,  and  z) 
and  recorded  it  in  the  seeding  log. 

Surface  Clearance  Range 

After  surveying,  the  surface  clearance  range  was  seeded  with  400  pieces  of  material, 
including  small  plate,  angle  iron,  and  small,  medium,  and  large  ISOs.  Each  seed  item 
was  painted  in  a  high-visibility  color  for  easier  visual  identification.  Figure  10  shows  the 
surface  clearance  range  with  the  ARTS  towing  a  Cherrington  beach  cleaner  attachment. 


Figure  10.  A  Picture  of  the  Surface  Clearance  Range. 


90 

Distribution  A:  Approved  for  public  release;  distribution  unlimited.  88ABW-2012-5009,  19  Sep  2012. 


52j#  Robotic  Ranse 

Clearance  Competition  R2C2  Validation  Trials  AAR  and  Rules  Changes  v.  28  MAY  2011 

Sub-Surface  Clearance  Range 

After  surveying,  the  range  was  seeded  with  20  small,  20  medium,  and  20  large  ISOs. 
The  pattern  was  arbitrarily  chosen  as  two  notional  target  points  with  seeded  items 
radiating  out  from  each.  Each  ISO  location  was  buried  according  to  the  seeding  plan  and 
each  location  was  surveyed  and  logged.  Figure  11  shows  the  ARTS  being  teleoperated 
on  the  sub-surface  clearance  range. 


Figure  11.  The  ARTS  on  the  Sub-Surface  Range  Digging  for  ISOs. 


Vegetation  Clearance  Range 

For  the  purposes  of  the  validation  trials,  the  real  vegetation  clearance  range  was  divided 
so  that  a  section  dominated  by  trees  and  another  section  was  all  high  prairie  grass  and 
very  small  vegetation.  The  area  with  the  trees  was  the  westernmost  area  of  the  range, 
while  the  area  with  grass  was  east,  shown  in  Figure  12.  The  grassy  area  was  created  to 
demonstrate  mowing,  while  the  treed  area  was  created  to  demonstrate  tree  felling  and 
mulching. 


Figure  12.  Real  Vegetation  Clearance  Range  is  Southernmost  Range. 

For  the  treed  area,  trees  were  thinned  so  that  each  tree  to  be  cut  was  separated  by  the 
next  tree  by  approximately  10  meters.  This  was  done  to  demonstrate  range 
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standardization  and  to  prevent  downed  trees  from  hanging  up  on  other  trees, 
complicating  the  tree  cutting  process.  Figure  13  shows  the  GyroTrac  cutting  trees  on  the 
vegetation  clearance  range’s  western  portion. 


Figure  13.  The  GyroTrac  Cutting  Trees  on  the  Vegetation  Clearance  Range. 
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Validation  Trials 

The  University  of  Florida/ARA  team  acted  as  the  notional  competitor.  The  plan  was  to 
demonstrate  some  level  of  performance  in  all  events  in  an  effort  to  exercise  the 
competition  plan  and  to  test  the  scoring  plan. 

The  UF/ARA  team  had  a  mix  of  autonomy  levels.  For  geophysical  mapping,  they  used  a 
fully  autonomous  ARTS  towing  the  ARA/NED  UXO  Trailer  described  earlier  and  shown 
in  Figure  14  below. 


Figure  14.  ARTS  Towing  the  UXO  Trailer  During  the  Geophysical  Mapping  Event. 


For  sub-surface  clearance,  they  used  a  mostly  tele-operated  ARTS  with  a  backhoe 
attachment,  using  autonomy  to  drive  to  the  dig  locations,  but  tele-operating  the  digging 
behavior  (shown  in  Figure  11). 

For  surface  clearance,  they  used  a  semi-autonomous  ARTS  system  towing  a 
Cherrington  beach  cleaner  (see  Figure  15).  The  ARTS  navigation  and  steering  behavior 
was  autonomous,  but  the  beach  cleaner  behavior  was  tele-operated. 


raBsiH 


Figure  15.  ARTS  with  the  Cherrington  Beach  Cleaner  Attachment. 
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And  finally,  for  vegetation  clearance,  the  UF/ARA  team  used  a  fully  autonomous  ARTS 
pushing  a  mowing  deck  for  grass  and  small  vegetation  and  a  tele-operated  Gyro-Trac 
mulching  machine  for  trees. 

Video  Monitoring/Boundary  Instrumentation 

Four  cameras  mounted  on  posts  monitored  three  edges  of  the  active  range  out-of- 
bounds  as  depicted  in  Figure  16.  The  fourth  edge  was  designated  as  the  entrance/exit  to 
the  range  and  was  not  monitored.  This  entrance/exit  area  is  a  subset  of  the  Range  thus 
competitors  will  not  be  penalized  if  they  enter  this  area. 

For  the  competition  we  will  add  an  additional  Pan,  Tilt,  Zoom  (PTZ)  Camera  that  will  be 
placed  outside  of  the  Area  of  Operations  to  monitor  competitor  progress  and  ensure 
potential  safety  hazards  are  quickly  addressed  during  the  competition. 


Camera 

X  X 


Figure  16.  Camera  Positions  for  the  Range  Boundary  Monitoring  System. 

Range  out-of-bounds  posts,  8  foot  PVC  pipe,  were  placed  in  line  with  the  camera  as  a 
boundary  reference  point  as  marked  in  Figure  17. 


Figure  17.  Out  of  Bounds  Posts  as  Shown  on  Video  Image. 
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Range  operations  and  boundary  violations  were  monitored  and  recorded  on  a  DVR. 

After  a  violation  was  identified,  a  visual  verification  and  event  documentation  was 
performed.  Figure  18  shows  one  of  the  boundary  violations  recorded  during  the 
geophysical  mapping  event. 

During  the  competition  painted  lines  will  be  added  to  each  range  to  more  easily  denote 
the  range  boundaries,  AO  boundaries,  and  penalization  boundary  to  allow  definitive  Q&A 
results  to  potential  AO  violations.  In  addition  we  will  explore  the  use  of  video  display 
overlays  to  mark  boundaries  for  easier  boundary  violation  judging. 


Figure  18.  A  Boundary  Violation  is  Visible  in  this  Scene. 

We  plan  on  using  the  same  boundary  instrumentation  in  the  actual  competition  as  that 
used  during  the  validation  trials,  but  with  better  resolution  cameras  and  DVR  to  improve 
the  video  quality  and  our  ability  to  make  these  critical  judgments. 

Summary  Results  of  Each  Trial 

Each  trial  will  be  summarized  to  include  the  system  performance,  level  of  human 
interaction,  man  hours  to  perform,  total  time  on  the  range,  and  relevant  comments 
characterizing  the  competitor  event  performance.  In  addition,  the  competitor’s  score 
sheet  will  be  presented  for  each  trial. 

The  overall  score  of  our  notional  competitor  team  was  547.9  out  of  1000  possible  points. 
This  score  and  its  breakdown  by  event,  shown  in  Figure  19,  comes  with  some  caveats. 


Notional  Competitor  Overall  Score 

547.9 

Vegetation  Clearance 

152.5 

Surface  Clearance 

113.5 

Geophyiscal  Mapping 

238.0 

Sub-surface  Clearance 

43.9 

Figure  19.  Overall  Notional  Competitor  Score  Sheet. 

First,  none  of  the  validation  trial  events  were  completed.  Once  a  general  performance 
pattern  was  established  and  enough  data  were  collected  to  evaluate  our  scoring  criteria, 
we  halted  events.  As  individual  event  score  sheets  show,  the  longest  time  spent  during 
any  event  was  six  hours.  Since  we  cap  event  durations  for  each  team  in  the  actual 
competition  at  eight  hours,  we  would  expect  that  these  notional  scores  would  only 
increase.  Each  score  sheet  shows  the  separate  score  buildups  by  category  (explained 
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more  in  each  following  section.)  They  also  show  competiton  event-level  metrics  in  the 
form  of  violations.  For  example,  perimeter  violations  are  worth  50  points  each,  each 
surface  damage  penalty  and  surface  grade  penalty  were  worth  1  point  each  (see  Rules 
document  for  more  explanation.) 

Geophysical  Mapping  Trial 

Scores  and  Comments 

The  event  score  was  238  out  of  250  possible  (see  Figure  20.)  This  represents  the 
revised  score  after  removing  the  boundary  violations  (which  would  have  made  the  actual 
score  less  than  negative  2,000,  with  each  boundary  violation  counting  as  a  50  point 
penalty.) 


Geophysical  Mapping  Event  Score 

238.02 

System  Task  Performance 

226.05 

50% 

Level  of  Human  Interaction 

250 

40% 

Man  Hours  to  Perform  the  Task 

250 

10% 

Competition  Level  Metrics 

Points/violation 

Violations 

Total 

Perimeteror  No-go  area  violation 

50 

0 

0 

Exclusion  zone  violation 

DQ 

Surface  Damage  (>15  cm) 

1 

0 

0 

Surface  Grade  (>15  cm) 

1 

0 

0 

Total  Point  Penalty 

0 

System  Task  Performance  (250  points) 

226.05 

Possible  points 

Violations 

Total 

Delivery  of  Raw  Data  (correct  format  within  48  hrs) 

250 

0 

0 

Noise  Level 

70 

70 

Sensor  Coverage 

60 

57 

Anomaly  Location  Accuracy 

60 

59.05 

Survey  Speed 

60 

40 

Design  Speed 

0.6 

m/s 

Level  of  Human  Interaction  (250  points) 

250 

Possible  points 

Description  of  Level  of  Human  Interaction 

250 

Human  Interaction  Pet 

Near  zero  human  interaction 

100% 

100% 

Minimal  Interaction 

75% 

Moderate  Interaction 

50% 

Frequent  Interaction 

25% 

Continuous  Interaction  (completely  tele-operated.) 

0% 

Man  Hours  to  Perform  the  Task  (250  points) 

250 

Start  Time 

End  Time 

Elapsed  Time 

4:58:06 

12:21:24 

17:19:30 

Numberof  people  on  site 

2 

Total  Man-Hours 

9.94 

Man  Hours  Allotted 

16 

Figure  20.  Notional  Competitor  Geophysical  Mapping  Event  Score  Sheet. 
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System  Performance 

The  system  task  performance  score  was  based  on  initial  data  analysis,  but  is  close 
enough  to  what  the  actual  score  would  have  been  for  the  purposes  of  this  analysis.  The 
only  reason  that  sensor  coverage  and  anomaly  detection  scores  were  not  higher  was 
that  the  trial  was  halted  before  the  system  had  managed  to  completely  cover  the 
mapping  range. 

Level  of  Human  Interaction 

The  Level  of  Human  Interaction  was  determined  to  be  near  zero,  so  100%  of  those 
points  were  awarded.  There  was  actually  some  amount  of  human  interaction,  but  most 
of  it  can  be  attributed  to  not  wanting  to  damage  the  UXO  Trailer.  So  even  though  there 
were  approximately  70  human  interactions  during  this  event,  their  total  duration 
accounted  for  only  1%  of  the  event’s  elapsed  time.  Also,  most  of  these  interactions 
happened  very  early  during  the  event,  while  the  team  was  being  cautious  about  the 
system  and  taking  time  to  build  trust. 

Man  Hours  to  Perform 

Man  Hours  were  calculated  as  9.94  (2  people  times  4  hours  58  minutes  elapsed  time) 
out  of  16  baseline  hours,  so  the  maximum  amount  of  man  hour  points  were  awarded. 
Baseline  man  hours  for  each  event  were  assumed  to  be  16  hours  (Maximum  event  time 
of  8  hours  multiplied  by  2  people:  Two  people  are  assumed  to  be  the  minimum  required 
for  safety.)  The  current  rules  calculate  the  man  hours  score  as  awarding  the  maximum 
man  hours  score  to  any  man  hours  total  at  or  below  the  baseline  and  greater  man  hours 
amounts  as  proportionally  fewer  points.  Since,  functionally,  the  maximum  number  of 
man  hours  points  that  are  awarded  at  the  event  level  is  25  (10%  of  250),  the  overall 
impact  of  changes  in  man  hours  is  relatively  small. 
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Sub-Surface  Removal  Trial 


Scores  and  Comments 

The  event  score  was  43.9  out  of  250  possible  (see  Figure  21 .)  This  event  was  done 
using  autonomy  to  move  the  vehicle  into  digging  position  over  the  dig  location,  but 
completely  tele-operated  digging  behavior.  A  number  of  problems  complicated  the 
teleoperation  aspect.  There  was  high  video  latency  and  poor  video  quality  (frame  rate, 
resolutions,  poor  gain  resulting  in  extreme  lighting  contrast,  and  compression  artifacts) 
as  a  result  the  digging  operation  was  hard  to  control  and  the  ISOs  were  hard  to  see  in 
the  video,  and  so  progress  was  slow.  Small  ISOs  were  used  in  addition  to  medium  and 
large  ISOs. 


Sub-surface  Clearance  Event  Score 

43.90 

System  Task  Performance 

12.5 

50% 

Level  of  Human  Interaction 

62.5 

40% 

Man  Hours  to  Perform  the  Task 

166.5 

10% 

Competition  Level  Metrics 

Points/violation 

Violations 

Total 

Perimeter  or  No-go  area  violation 

50 

0 

0 

Exclusion  zone  violation 

DQ 

Surface  Damage  (>15  cm) 

1 

0 

0 

Surface  Grade  (>15  cm) 

1 

4 

4 

Total  Point  Penalty 

4 

System  Task  Performance  (250  points) 

12.5 

Possible  points 

Number 

removed 

Possible 

seeds 

Sub-surface  Seed  Removal 

250 

3 

60 

Level  of  Human  Interaction  (250  points) 

62.5 

Possible  points 

Description  of  Level  of  Human  Interaction 

250 

Human  Interaction  Pet 

Near  zero  human  interaction 

100% 

25% 

Minimal  Interaction 

75% 

Moderate  Interaction 

50% 

Frequent  Interaction 

25% 

Continuous  Interaction  (completely  tele-operated.) 

0% 

Man  Hours  to  Perform  the  Task  (250  points) 

166.46 

Start  Time 

End  Time 

Elapsed  Time 

6:00:27 

12:51:18 

18:51:45 

Number  of  people  on  site 

4 

Total  Man-Hours 

24.03 

Man  Hours  Allotted 

16 

Figure  21.  Notional  Competitor  Sub-Surface  Clearance  Event  Score  Sheet. 
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System  Performance 

The  system  task  performance  score  was  12.5  points  based  on  recovering  3  out  of  60 
possible  seed  items.  The  vehicle  navigation  behavior  was  extremely  accurate,  but  two 
factors  contributed  to  less  than  optimal  digging.  One  was  that  the  system  did  not  provide 
any  feedback  on  the  position  of  the  backhoe,  so  it  was  difficult  for  the  tele-operator  to 
always  accurately  position  the  bucket. 

Level  of  Human  Interaction 

The  Level  of  Human  Interaction  was  determined  to  be  frequent,  so  25%  of  those  points 
were  awarded.  The  vast  majority  of  the  event  duration  time  was  spent  tele-operating  the 
bucket.  When  digging  for  a  particular  ISO  was  completed,  the  vehicle  was  put  into  an 
autonomous  navigation  mode  where  it  moved  to  the  pit  area  to  dump  the  ISO  and  then 
autonomously  navigated  to  the  next  dig  location. 

Man  Hours  to  Perform 

Man  Hours  were  calculated  as  24  (4  people  times  6  hours  elapsed  time)  out  of  16 
baseline  hours,  so  only  166  of  the  maximum  man  hour  points  were  awarded.  Baseline 
man  hours  for  each  event  were  assumed  to  be  16  hours  (Maximum  event  time  of  8 
hours  multiplied  by  2  people).  Two  people  are  assumed  to  be  the  minimum  required  for 
safety.  In  this  case,  additional  people  were  needed  to  troubleshoot  system  problems,  so 
their  presence,  no  matter  how  brief,  was  added  to  the  team’s  total. 
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Surface  Clearance  Trial 


Scores  and  Comments 

The  event  score  was  1 13.5  out  of  250  possible  (see  Figure  22.)  This  event  was 
performed  with  an  autonomously  navigated  ARTS  pulling  and  powering  a  Cherrington 
Beach  Cleaner  attachment.  Adjustments  to  the  beach  cleaner,  such  as  changing  the 
rake  position  or  dumping  the  bucket,  were  “tele-operated”,  in  that  they  were  activated 
from  the  command  trailer  by  operators  monitoring  the  system’s  progress  and  then 
activating  these  functions  with  key  presses. 


Surface  Clearance  Event  Score 

113.5 

System  Task  Performance 

27 

50% 

Level  of  Human  Interaction 

187.5 

40% 

Man  Hours  to  Perform  the  Task 

250 

10% 

Competition  Level  Metrics 
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Perimeter  or  No-go  area  violation 

50 

0 

0 

Exclusion  zone  violation 

DQ 
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1 

0 

0 
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1 

0 

0 

Total  Point  Penalty 

0 

System  Task  Performance  (250  points) 

27 

Possible  points 
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Percent  debris  removed 
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14 

Pile  cleanliness 
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13 

Level  of  Human  Interaction  (250  points) 

187.5 

Possible  points 

Description  of  Level  of  Human  Interaction 

250 

Human  Interaction  Pet 

Near  zero  human  interaction 

100% 

75% 

Minimal  Interaction 

75% 

Moderate  Interaction 

50% 

Frequent  Interaction 

25% 

Continuous  Interaction  (completely  tele-operated.) 

0% 

Man  Hours  to  Perform  the  Task  (250  points) 

250 

Start  Time 

End  Time 

Elapsed  Time 

2:10:25 

11:41:55 

13:52:20 

Numberof  people  on  site 

2 

Total  Man-Hours 

4.35 

Man  Hours  Allotted 

16 

Figure  22.  Notional  Competitor  Surface  Clearance  Event  Score  Sheet. 
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System  Performance 

The  system  task  performance  score  was  27  points  based  on  recovering  44  out  of  400 
possible  seed  items,  and  also  having  particularly  dirty  debris  piles  because  no  shaker 
table  was  used.  The  ARTS  seemed  a  little  underpowered  at  times  while  towing  the 
Cherrington  attachment,  and  the  large  ISOs  were  particularly  problematic,  jamming  the 
machine  several  times.  The  angle  iron  and  the  smaller  ISOs  were  picked  up  fairly  well, 
but  the  steel  plate  was  missed.  However,  these  performance  issues  may  have  been 
solved  with  more  practice  time  and  system  tuning.  The  system  suffered  a  mechanical 
failure  after  the  tool  attachment  face  of  the  ARTS  struck  the  Cherrington  hydraulic  pump. 
As  no  spares  were  on  hand,  this  event  ended  the  trial  after  significant  troubleshooting. 

Level  of  Human  Interaction 

The  Level  of  Human  Interaction  was  determined  to  be  minimal,  so  75%  of  those  points 
were  awarded.  The  vast  majority  of  operators’  time  was  spent  monitoring  the  system  for 
problems  and  when  attachment  adjustments  needed  to  be  made. 

Man  Hours  to  Perform 

Man  Hours  were  calculated  as  4.35  (2  people  times  2  hours  10  minutes  elapsed  time) 
out  of  16  baseline  hours,  so  the  maximum  man  hour  points  were  awarded.  Baseline  man 
hours  for  each  event  were  assumed  to  be  16  hours  (Maximum  event  time  of  8  hours 
multiplied  by  2  people).  Two  people  are  assumed  to  be  the  minimum  required  for  safety. 
In  terms  of  the  overall  elapsed  time,  a  significant  portion  was  dedicated  to 
troubleshooting  the  system. 
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Vegetation  Clearance  Trial 

Scores  and  Comments 

The  event  score  was  152.5  out  of  250  possible  (see  Figure  23.)  This  event  was 
performed  using  two  vehicles.  The  ARTS  was  used  to  autonomously  position  a  mowing 
deck  over  the  portion  of  the  range  that  was  only  tall  grass  and  very  small  shrub.  The 
Gyro-Trac  mulching  machine,  in  a  complete  tele-operation  mode,  was  used  to  cut  trees 
and  mulch  them  over  the  portion  of  the  range  with  trees.  The  score  was  the  combined 
score  of  the  two  phases. 


Vegetation  Clearance  Event  Score 

152.5 

System  Task  Performance 
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50% 

Level  of  Human  Interaction 

125 

40% 

Man  Hours  to  Perform  the  Task 
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10% 

Competition  Level  Metrics 
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Total 
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0 

0 
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1 

0 

0 
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1 

0 

0 
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0 
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34 

40 
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4 
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Level  of  Human  Interaction  (250  points) 
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Description  of  Level  of  Human  Interaction 
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Human  Interaction  Pet 

Near  zero  human  interaction 

100% 

50% 

Minimal  Interaction 

75% 

Moderate  Interaction 

50% 

Frequent  Interaction 

25% 

Continuous  Interaction  (completely  tele-operated.) 

0% 

Man  Hours  to  Perform  the  Task  (250  points) 
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Start  Time 

End  Time 

Elapsed  Time 

1:41:00 

10:15:00 

11:56:00 

Number  of  people  on  site 

2 

Total  Man-Hours 

3.37 

Man  Hours  Allotted 

16 

Figure  23.  Notional  Competitor  Vegetation  Clearance  Event  Score  Sheet. 


System  Performance 

The  system  task  performance  score  was  155  points  based  on  having  successfully 
mowed  half  of  the  range  and  having  cut  down  1 6  of  50  trees  on  the  range,  but  only 
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successfully  mulching  12  of  the  16  trees.  Had  the  mowing  not  been  as  successful,  with 
swaths  left  unmowed,  the  score  would  have  been  reduced  vegetation  residue  removal 
penalties. 

Level  of  Human  Interaction 

Since  the  “mowing”  portion  was  done  using  no  human  interaction  and  the  tree  cutting 
portion  was  done  completely  tele-operated,  the  aggregate  level  of  human  interaction 
was  determined  to  be  50%.  During  the  mowing  phase,  there  was  one  human  interaction 
with  the  system  to  change  the  speed  of  the  machine. 

Man  Hours  to  Perform 

Man  Hours  were  calculated  as  3.37  (2  people  times  1  hour  41  minutes  elapsed  time)  out 
of  16  baseline  hours,  so  the  maximum  man  hour  points  were  awarded.  Baseline  man 
hours  for  each  event  were  assumed  to  be  16  hours  (Maximum  event  time  of  8  hours 
multiplied  by  2  people).  Two  people  are  assumed  to  be  the  minimum  required  for  safety. 
It  was  assumed  that  the  two  different  phases  occurred  simultaneously.  They  did  not 
during  the  trials,  but  the  score  would  not  change  appreciably  regardless. 
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Analysis  to  Illustrate  Different  Levels  of  Autonomy 

To  give  the  reader  a  sense  of  how  different  autonomy  levels  change  event  scores,  see 
Figure  24.  The  chart  illustrates  the  changes  in  event  score  as  a  function  of  autonomy 
level,  ranging  from  completely  tele-operated  (0%),  moving  to  a  more  balanced 
competitor  solution  (50%  autonomy),  to  a  nearly  completely  autonomous  solution 
(100%). 


250.0 
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150.0 
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Vegetation  Surface  Geophysical  Sub-surface 
Clearance  Clearance  Mapping  Clearance 
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Competitor 

■  Ideal  Autonomous 
Competitor 


Figure  24.  Event  Score  as  a  Function  of  Autonomy  Level. 

These  scores  were  calculated  using  the  system  performance  and  man  hours  scores  that 
were  found  during  the  validation  trials.  Under  the  current  scoring  scheme,  moving  from  a 
0%  autonomy  score  to  a  50%  autonomy  score  results  in  a  50  point  increase.  The  same 
increase  occurs  moving  from  50%  to  100%  autonomy.  What  is  not  captured  is  how  an 
increase  in  autonomy  might  enable  a  team  finishing  sooner,  or  more  significantly, 
attaining  a  higher  system  performance  score  as  a  result.  But  it  is  clear  that  higher  levels 
of  autonomy  will  result  in  significantly  higher  scores,  even  if  performance  is  unaffected. 
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Outcomes 

Validated  Range  Construction  Approach  and  Logistics 

Finding:  Facilities  and  competition  ranges  were  appropriate  for  this  test. 
Recommendation:  Two  hard  sided  "worksite  trailers"  will  be  provided  for  the  support  of 
competition  management  and  distinguished  visitors.  One  trailer  will  be  for  the  R2C2 
mgmt/support  team  the  second  trailer  will  be  reserved  to  support  business  needs  of  the 
JGRE  director,  Chief  Judge,  and  any  distinguished  visitors  (i.e.  military  commanders/ 
generals,  congressional  personnel,  etc.).  The  management  trailer  configuration  will  be 
selected  by  JTEC  personnel.  The  Distinguished  Visitor  (DV)  trailer  should  have  at  least 
two  private  offices  and  a  conference  area  with  comfortable  chairs,  internet  connections, 
and  a  couch. 

Validated  Scoring  Criteria 

Finding:  The  range  perimeter  established  for  this  test  was  too  restrictive  and  resulted  in 
numerous  (~45)  excursions  (penalized  at  50  points  each). 

Recommendation:  The  setup  of  the  competition  areas  will  be  modified  so  that  the  area 
of  operation  will  be  defined  as  30  meters  around  the  perimeter  of  the  target  area.  The 
penalty  area  will  be  3  meters  outside  of  this  area  of  operation. 

Additional  Recommendations 

The  following  represent  additional  recommendations  for  the  actual  competition: 

•  The  competitors  will  be  given  the  opportunity  to  arrive  at  the  competition  1  week 
prior  to  the  start.  They  will  be  provided  practice  areas  to  fine  tune  and  debug  any 
last  minute  issues  or  failures  of  their  systems  during  this  time.  For  the 
competition  they  will  then  have  their  assigned  time  to  complete  the  competition 
task.  The  only  exceptions  to  this  will  be  for  weather,  safety,  or  military  priority 
issues  that  might  delay  or  impede  the  competition  activity. 

•  The  dig  map  provided  for  the  Sub-Surface  UXO  clearance  tasks  will  include  a 
"target  strength  value"  to  represent  a  realistic  geophysical  mapping  product.  The 
makeup  of  this  value  will  be  determined  through  consultation  with  Mr.  Andy 
Schwartz,  USACOE  Geophysicist.  In  addition  to  assigning  a  weighted  value  to 
UXOs,  we  will  not  include  the  “small”  type  ISOs  in  Sub-Surface  clearance 
ranges.  However,  we  will  keep  the  total  number  of  ISOs  at  50-60,  or  25-30  Large 
ISOs  and  30  Medium  ISOs. 

•  A  meeting  between  the  JGRE  director  and  Camp  Guernsey  Garrison 
Commander  is  suggested  to  discuss  importance  of  this  activity  and  commitment 
for  a  guaranteed  level  of  lodging  and  support. 

•  The  Oversight  IPT  should  review  the  categories  of  human  interaction  to  ensure 
that  they  reflect  the  intent  of  the  competition.  Currently,  any  automated  behavior 
at  all,  no  matter  how  little,  results  in  25%  of  the  available  human  interaction 
points. 
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Recommended  Changes  to  Competition  Rules 

The  outcomes  of  the  validation  trials  in  October  suggest  that  the  following  changes  to 
the  rules  be  considered. 


Area  of  Operation  Boundary 

The  Area  of  Operation  Boundary  will  be  defined  as  being  30  meters  beyond  the  range 
boundary  and  the  language  in  the  Rules  document  will  be  changed  for  consistency  of 
terminology. 

This  change  results  from  the  number  of  boundary  excursions  that  occurred  during  the 
geophysical  mapping  trials.  The  Notional  competition  team  “lost”  2,050  points  due  to 
their  excursions  (a  50  point  penalty  for  each),  resulting  in  a  net  negative  score.  It  was 
determined  that  the  boundary  was  too  restrictive  as  it  was  operationalized  for  the 
validation  trials.  A  larger  boundary  to  allow  machinery  to  turn  around  with  incurring  a 
penalty  has  been  recommended. 


We  will  add  a  graphic  to  the  rules  document  that  assists  in  term  clarification  as  shown  in 
Figure  25. 


Penalty  Boundary 
(3m  from  AO  Boundary) 

AO  Boundary 

(30m  from  Range  Boundary) 
-  Range  Area 


Figure  25.  Graphical  Depiction  of  different  areas. 


Surface  Clearance  QC  Pickup 

In  Surface  Clearance  section,  rule  “A.c”  will  be  changed  from  referring  to  “weight”  of  the 
QC  pickup  to  “number”.  This  will  greatly  simplify  the  QC  check  for  this  event. 

The  current  rule  states  that  the  material  in  the  QC  pickup  will  be  weighed  to  determine 
the  performance  penalty.  However,  given  the  number  of  pieces  and  their  high-visibility 
color,  it  was  determined  that  it  will  be  easier  to  count  the  pieces  than  it  will  be  to 
accurately  weigh  them. 

Standoff  Range 

Currently,  the  Rules  document  states  that  all  competitors  must  remain  outside  of  the 
exclusion  zone  during  an  event,  which  is  870  meters.  This  distance  is  impractical  to 
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enforce  during  the  competition  due  to  site  size  considerations.  We  will  modify  the  rules 
document  to  state  that  all  systems  must  be  capable  of  operating  at  870  meters,  to 
simulate  the  minimum  safe  standoff  distance  for  large  artillery  disposal  operations,  but 
during  the  competition,  competitor  personnel  must  remain  outside  the  Area  of  Operation. 
Further  the  term  “Exclusion  Zone”  section  in  the  Rules  and  Metrics  Document  should  be 
changed  to  “Standoff  Capability  Parameters”  to  better  reflect  the  intent  of  this  section. 

Continual  Effort 

During  an  event,  each  Competitor  Team  must  make  a  continual  effort  to  complete  the 
event  until  the  goal  or  the  8-hour  time  limit  has  been  reached.  If  a  Competitor  T earn 
stops  attempting  to  make  progress  toward  the  event  goal,  their  score  for  the  entire  event 
will  be  zero.  A  continual  effort  shall  constitute  any  and  all  efforts  to  fix,  repair,  rework  or 
brainstorm  solutions  to  non-functioning  equipment.  If  it  is  determined  by  an  R2C2 
observer  that  continual  effort  is  not  being  made  the  observer  will  inform  the  competitor’s 
team  leader  by  giving  one  warning.  If  the  observer  does  not  see  efforts  resumed 
immediately,  the  competitor  shall  be  given  a  score  of  zero  for  the  entire  event. 

The  purpose  of  this  suggested  rule  change  is  to  prevent  teams  from  preserving  their 
man  hours  and  human-interaction  scores  by  not  performing.  Currently,  the  rules  do  not 
preclude  a  competitor  from  simply  declaring  that  they  have  begun  the  competition  and 
then  immediately  declaring  they  have  completed  the  competition  without  actually  doing 
anything.  If  a  competitor  were  to  do  that  under  the  current  rules,  they  could  actually 
receive  125  points  out  of  250  for  any  event  they  entered.  This  strategy  needs  to  be 
discouraged  as  it  does  not  meet  the  intent  of  the  competition. 

Theshold  Performance 

It  is  expected  that  for  any  competitors  to  win  an  event,  they  must  have  completed  a 
certain  portion  of  that  event  to  ensure  that  the  JGRE  can  be  satisfied  that  winners  are 
recognized  for  having  performed  reasonably  well,  and  not  just  less  poorly  than  other 
competitors  (i.e. ,  no  winners  by  default.)  Therefore,  in  order  to  be  eligible  for  prize 
money  in  any  event,  competitors  will  have  had  to  complete  at  least  25%  of  the  goal  for 
that  event. 


Vegetation  Clearance 

A  rule  change  is  recommended  in  vegetation  clearance  scoring  to  count  any  tree  that  is 
not  cut  as  a  tree  removal  penalty  and  also  as  a  residue  removal  penalty.  The  intent  of 
this  rule  change  is  to  put  more  emphasis  on  cutting  trees  and  less  emphasis  on  keeping 
mulch  to  a  manageable  depth. 

Currently,  the  scoring  for  vegetation  clearance  is  equally  weighted  (125  points)  between 
tree  cutting  and  mulch  depth  management.  However,  consider  two  teams  that  both 
maximize  their  human  interaction  score  and  their  man  hours  score.  One  of  the  teams 
cuts  all  of  its  trees  down,  but  gets  penalized  an  equal  number  of  points  for  failure  to  keep 
mulch  under  the  maximum  15cm  depth.  That  team’s  score  would  be  187.5  under  the 
current  rules. 


This  is  an  illustration  of  the  impact  of  the  current  scoring  system  on  Vegetation 
Clearance.  Both  score  sheets  assume  perfect  scores  for  human  interaction  and  man 
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hours.  The  first  score  (187.5),  in  Figure  26,  represents  cutting  all  trees,  but  receiving  125 
points  in  penalties  for  leaving  mulch  at  too  great  a  depth.  The  second  score  (203.125),  in 
Figure  27,  represents  cutting  only  25%  of  the  trees,  but  no  residue  penalty. 


Vegetation  Clearance  Event  Score 

187.5 

System  Task  Performance 

125 

50% 

Level  of  Human  Interaction 

Man  Hours  to  Perform  the  Task 

250 

250 

40% 

10% 

Competition  Level  Metrics 

Points/violation  Violations 

Total 

Perimeter  or  No  go  area  violation 

50  0 

0 

exclusion  zone  violation 

DQ 

Surface  Damage  (>15cm) 

1 

0 

0 

Surface  Grade  (>15cm) 

1 

0 

0 

Total  Point  Penalty 

0 

System  Task  Performance  (250  points) 

125 

Possible  points 

Violations 

Vegetation  Tree  Removal  (>8cm) 

125  0 

125 

Vegetation  Residue  Removal  (>15cm) 

125  125 

0 

Figure  26.  Current  Scoring  System  -  Cutting  All  Trees  with  Maximum  Residue  Penalty. 


Vegetation  Clearance  Event  Score 

203.125 

System  Task  Performance 

156.25 

50% 

Level  of  Human  Interaction 

250 

40% 

Man  Hours  to  Perform  the  Task 

250 

10% 

Competition  Level  Metrics 

Poi  nts/violation 

Violations 

Total 

Perimeter  or  No  go  area  violation 

50 

0 

0 

Exclusion  zone  violation 

DQ 

Surface  Damage  (>15cm) 

1 

0 

0 

Surface  Grade  (>15  cm) 

1 

0 

0 

Total  Point  Penalty 

0 

System  Task  Performance  (250  points) 

156. 25 

Possible  points 

Violations 

Vegetation  Tree  Removal  (>8cm) 

125 

93.75 

31.25 

Vegetation  Residue  Removal  (>15cm) 

125 

0 

125 

Figure  27.  Current  Scoring  System  -  Cutting  25%  Trees  with  No  Residue  Penalty. 

So,  under  the  current  scoring  scheme,  cutting  %  of  the  trees  perfectly  results  in  a  higher 
score  than  cutting  all  of  the  trees  imperfectly. 
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There  are  at  least  two  ways  to  handle  this.  1)  Consider  a  non-felled  tree  as  not  only  a 
tree  removal  penalty,  but  also  a  residue  penalty.  That  way,  there  is  no  advantage  to 
leaving  trees  standing  while  focusing  on  mulching.  If  a  competitor  only  cut  25%  of  the 
trees  in  this  case,  they  would  only  receive  a  total  of  25%  of  the  System  Task 
Performance  points,  resulting  in  a  maximum  possible  total  event  score  of  156.25  (shown 
in  Figure  28.) 


Vegetation  Clearance  Event  Score 

156.25 

System  Task  Performance 

62.5 

50% 

Level  of  Human  Interaction 

250 

40% 

Man  Hours  to  Perform  the  Task 

250 

10% 

Competition  Level  Metrics 

Points/violation 

Violations 

Total 

Perimeter  or  No-go  area  violation 

50 

0 

0 

Exclusion  zone  violation 

DQ 

Surface  Damage  (>15  cm) 

1 

0 

0 

Surface  Grade  (>15  cm) 

1 

0 

0 

Total  Point  Penalty 

0 

System  Task  Performance  (250  points) 

62.5 

Possible  points 

Violations 

Vegetation  Tree  Removal  (>8cm) 

125 

93.75 

31.25 

Vegetation  Residue  Removal  (>15cm) 

125 

93.75 

31.25 

Figure  28.  Proposed  Rule  Clarification  -  Cutting  25%  Trees  with  Equal  Tree  and  Residue 

Penalty. 


2)  Change  the  point  distribution  between  clearance  and  mulching.  One  possibility  is 
to  change  the  point  values  so  that  tree  removal  is  worth  200  points  and  the  maximum 
residue  penalty  is  50  points. 

In  that  scoring  scenario,  the  same  performance  situation  results  in  a  50  point  advantage 
for  cutting  all  trees  versus  cutting  only  25%  of  the  trees  with  no  mulch  penalty.  Figures 
29  and  30  show  the  scoring  illustration. 

It  should  be  noted  that  this  scenario  is  only  valid  if  the  maximum  mulch  (residue)  penalty 
is  capped.  If  the  penalty  can  be  greater,  then  this  effect  is  lessened.  The  reason  is  that  in 
this  rules  interpretation,  there  can  only  be  a  residue  penalty  for  a  tree  that  has  been  cut. 
Any  tree  not  cut  will  not  count  against  the  residue  penalty  part  of  the  score.  So  it  will  not 
hurt  a  competitor  to  leave  trees  (besides  the  existing  tree  cutting  penalty). 
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Vegetation  Clearance  Event  Score 
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0 
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0 

0 

Surface  Grade  (>15  cm) 

1 

0 

0 
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0 
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Vegetation  Tree  Removal  (>8cm) 
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0 

200 

Vegetation  Residue  Removal  (>15cm) 

50 

50 

0 

Figure  29.  Proposed  New  Scoring  -  Cutting  Ail  Trees  with  Maximum  Residue  Penalty 

(Point  Value  Change). 


Vegetation  Clearance  Event  Score 
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System  Task  Performance 
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50% 

Level  of  Human  Interaction 
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40% 
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Points/violation 
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50 

0 

0 
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DQ 
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1 

0 

0 

Surface  Grade  (>15  cm) 

1 

0 

0 

Total  Point  Penalty 

0 

System  Task  Performance  (250  points) 

100 

Possible  points 

Violations 

Vegetation  Tree  Removal  (>8cm) 

200 

150 

50 

Vegetation  Residue  Removal  (>15  cm) 

50 

0 

50 

Figure  30.  Proposed  New  Scoring  -  Cutting  25%  Trees  with  No  Residue  Penalty 

(Point  Value  Change). 

Therefore,  to  simplify  any  rules  change,  the  final  recommendation  is  to  implement  the 
first  option  and  include  a  residue  penalty  if  a  tree  remains  standing.  This  change 
will  really  be  only  a  rules  clarification,  and  not  a  substantial  change. 
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Introduction 


Purpose 

The  purpose  of  the  work  plan  for  seeding  the  geophysical  mapping  range  is  to  ensure 
that  the  range  build  requirements  are  clearly  documented  and  that  the  process  for 
creating  the  range  and  burying  all  of  the  seed  items  is  clearly  understood  and  accepted. 

Scope 


This  document  will  describe  the  range  requirements  to  include  location,  size  and 
dimensions,  seed  items,  their  sizes  and  locations,  appropriate  burying  procedures,  range 
survey  procedures  and  appropriate  quality  control  checks. 

Objectives 

The  geophysical  mapping  range  being  built  in  the  North  Training  Area  (NTA)  of  Camp 
Guernsey,  WY  is  an  area  seeded  with  industry  standard  objects  (ISO)  acting  as 
surrogates  for  unexploded  ordnance  (UXO)  in  order  to  present  a  geophysical  mapping 
challenge  to  R2C2  competitors.  In  building  this  range  with  a  known  ground  truth  (i.e., 
precise  descriptions  and  locations  of  seeded  ISOs),  the  R2C2  can  evaluate  the 
effectiveness  of  competitors’  robotic  solutions  to  geophysical  mapping  challenges. 
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Figure  A-1.  Map  of  Wyoming  Showing  General  Location  of  R2C2  Ranges. 


The  Geophysical  Mapping  (GM)  Range 


The  range  to  be  constructed  has  an  area  of  approximately  2  acres,  located  in  the  North 
Training  Area  of  Camp  Guernsey,  WY.  Figure  A-2  shows  a  Google  Earth  image  of  the 
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Geophysical  Mapping  Range  to  be  constructed  and  also  shows  its  proximity  to  the  other 
ranges  being  constructed. 


Figure  A-2.  Locations  of  R2C2  Ranges  being  constructed  in  Camp  Guernsey  NTA. 
Location 

The  GM  range  is  the  northernmost  range  and  is  defined  as  a  four  sided  polygon  with 
corner  locations  defined  in  Table  A-1. 

Table  A-1.  Geophysical  Mapping  Range  Corner  Post  Coordinates 


Range  Name 

Location  ID 

MGRS  Coordinates 

Geophysical  Mapping 
Range 

NE  Corner 

13TEG1701598501 

NW  Corner 

13TEG1 690098445 

SE  Corner 

13TEG1 704898447 

SW  Corner 

13TEG1 694398390 

Range  Preparation  Process 

This  is  an  overview  description  of  the  range  preparation  process.  A  more  detailed 
description  of  surveying  and  QA  procedures  appears  at  the  end  of  this  document.  Range 
preparation  begins  with  a  precise  survey  of  all  four  corner  posts.  Their  approximate 
locations  are  given  in  Table  A-1 .  Corner  post  locations  will  be  established  within  the 
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range  in  Table  A-1,  marked  with  stakes,  and  the  stake  locations  precisely  surveyed. 
Corner  locations  will  then  be  recorded  in  a  table  with  their  location  identifier,  latitude  (in 
decimal  degrees),  longitude  (in  decimal  degrees),  and  elevation  (in  meters  above  the 
WGS84  datum.)  A  GPS-based  system  with  Real-Time  Kinematic  (RTK)  correction  will 
be  used  to  provide  accuracy  to  within  +/-  3  cm. 

The  next  step  in  the  preparation  process  is  for  the  range  to  be  mowed,  or  cleared  of  all 
vegetation  taller  than  8cm. 

We  will  then  randomly  place  obstacles  on  the  range.  The  obstacles  will  be  constructed  of 
railroad  ties. 

The  next  step  to  prepare  the  range  is  to  create  a  geophysical  baseline  map.  We  will  use 
the  UXO  Trailer  built  by  ARA’s  Northeast  Division  (NED)  as  the  baseline  mapping 
equipment. 

Once  the  geophysical  baseline  has  been  produced,  we  will  use  it  to  plan  the  seeding  of 
the  range  with  small  and  medium  industry  standard  objects  (ISO). 

The  final  step  is  to  dig  the  seed  locations,  place  the  ISOs,  survey  and  record  their 
locations  in  latitude,  longitude,  and  depth  in  centimeters,  and  bury  the  ISOs. 

Equipment 

The  survey  equipment  consists  of  an  Ashtech  Magellan  ProMark3  RTK  GPS  survey  kit. 
Figure  A-3  below  shows  the  kit  in  use. 


Figure  A-3.  Ashtech  Magellan  ProMark  3  RTK  Surveying  Equipment  in  Use. 

The  NED  UXO  Trailer  is  a  towed  trailer  featuring  a  Minelab  Single  Transmit  Multiple 
Receive  (STMR)  metal  detection  array.  The  detection  array  is  a  Time  domain 
electromagnetic  induction  sensor.  The  trailer  is  shown  in  Figure  A-4. 
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Figure  A-4.  The  NED  Basic  UXO  Trailer. 


Corner  Survey  Procedures 

We  will  use  a  crew  trained  in  the  use  of  the  Ashtech  Magellan  ProMark  3  RTK  system. 

The  project  manager  will  establish  the  actual  locations  of  the  corner  posts. 

The  survey  team  will  place  the  corner  posts  at  their  predetermined  locations  and  then 
survey  their  locations. 

The  surveying  team  will  ensure  that  their  measurements  are  precise  up  to  +/-  3  cm  in  the 
x,  y,  and  z  dimensions  by  allowing  enough  time  for  the  GPS  receiver  to  resolve  its 
location  to  a  sufficient  accuracy. 

The  survey  team  will  record  the  precise  locations  of  the  corner  posts. 

The  survey  team  will  then  reacquire  each  corner  post  location  and  verify  the 
documented  coordinates. 

Mowing  Process 

We  will  use  a  brush  cutter/  bush  hog  to  cut  down  all  of  the  vegetation  on  the  range  as 
low  to  the  ground  as  possible. 

Obstacle  Placement  Process 

Obstacles  will  be  constructed  of  railroad  ties.  Each  obstacle  will  be  constructed  of  10 
railroad  ties  -  a  bundle  of  2  by  5.  Obstacles  will  be  placed  at  various  locations  (to  be 
determined)  on  the  range. 

Geophysical  Mapping  Baseline  Procedures 

The  geophysical  mapping  baseline  will  be  completed  on  the  GM  range  after  the  corners 
of  the  range  have  been  surveyed,  but  before  the  seeding  has  been  done. 
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ARA’s  NED  will  collect  baseline  geophysical  data  for  the  entire  range  and  build  a 
geophysical  map  showing  the  electromagnetic  response  of  the  range. 

Baseline  data  will  be  collected  at  the  appropriate  speed  and  line  spacing  for  the  Minelab 
STMR  array. 

Data  will  be  collected  in  a  form  appropriate  for  post-processing  by  tools  such  as  Oasis 
Montaj. 

Collected  data  will  be  analyzed  for  average  electromagnetic  response  in  millivolts  and 
nanoTeslas. 

Localized  peaks  will  be  identified  for  marking  on  the  range  as  unacceptable  seed 
locations. 

Seeding  Procedures 

Using  the  analyzed  baseline  data,  the  project  manager  will  determine,  with 
recommendations  by  the  project  geophysicist,  the  approximate  locations  of  each  of  the 
63  medium  and  63  small  ISO  being  used  as  seeds. 

All  seed  locations  will  be  selected  so  that  no  other  anomaly,  either  pre-existing  or 
another  planned  seed,  will  be  within  2  meters  of  a  seed  location.  Since  the  mapping  area 
is  approximately  2  acres  this  means  about  1,875  potential  2m  x  2m  locations  exist  from 
which  we  need  only  find  100  to  125  that  are  "clean." 

All  seed  items  will  be  either  the  small  or  medium  ISO  (see  Figure  A-5.)  (Only  localized 
sensor  responses  are  needed  to  assess  how  well  robotic  systems  measure  where  EM 
data  are  collected.  Therefore,  large  ISOs  are  not  needed.) 


Figure  A-5.  Picture  of  ISOs  required  for  the  R2C2. 


The  ISOs  are  described  in  Table  A-2. 
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Table  A-2.  ISO  Descriptions  and  Specifications. 


Item 

Nominal 
Pipe  Size 

Outside 

Diameter 

Length 

Part 

Number1 

ASTM 

Specification 

Small  ISO 

1" 

1.315" 

(33  uun) 

4" 

(102  uun) 

44615K466 

A53/A773 

Medium  ISO 

2" 

2.375" 

(60  uun) 

8" 

(204  uun) 

44615K529 

A53/A773 

Large  ISO 

4" 

4.500" 

(115  mm) 

12" 

(306  mm) 

44615K137 

A53/A773 

1  Part  number  from  the  McMaster-Carr  catalog. 


All  seeds  will  be  buried  perpendicular  to  the  local  ground  surface  (i.e.  vertical  with 
respect  to  the  local  topography).  Coordinate  accuracy  will  be  +/-2cm  in  x,  y  and  z, 
vertical  orientation  will  be  accurate  to  +/-5  degrees.  These  burial  parameters  will  result  in 
strong,  single,  monopole  responses  from  horizontal  loop  EM  systems  such  as  the  EM61 
MK2. 

All  seeds  will  be  placed  in  the  ground  at  a  depth  of  5  to  7  times  their  diameter.  All  depth 
measurements  will  be  to  the  center  of  the  items.  This  may  require  that  for  each  size  of 
ISO,  an  offset  representing  the  distance  from  the  end  of  the  ISO  (the  top,  when  buried 
vertically)  to  its  center,  is  added  to  each  survey  measurement,  if  that  measurement  is 
taken  at  the  top  of  each  ISO. 


Figure  A-6.  An  ISO  Being  Surveyed  in  its  Burial  Location. 


(Obstacles  will  be  emplaced  on  the  range.  The  R2C2  will  identify  how  close  competitors 
are  expected  to  get  to  these  obstacles.  No  ISOs  will  be  seeded  within  the  buffer 
surrounding  these  obstacles.) 

The  burial  team  will  ensure  that  the  ISOs  are  buried  at  the  appropriate  depth  by  double 
checking  that  the  ISO  for  burial  is  the  appropriate  size  and  that  the  depth  is  5  to  7  times 
the  diameter  of  the  ISO. 

The  burial  team  will  survey  the  burial  location  (x,y,  and  z)  and  record  it  in  the  log 
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Description  of  Validation  Trials  (Surveying) 

Survey  Equipment  Setup 
Ad  Hoc  Monument 

Repeated  operation  of  the  GPS  system  requires  precise  placement  of  the  Base  station. 

If  the  Base  station  is  moved  between  samples,  the  measured  points  will  be  offset  from 
existing  points.  To  accurately  place  the  Base  station,  we  drove  a  pair  of  stakes  into  the 
ground  to  mark  the  position  of  the  Base  station.  The  point  at  the  bottom  of  the  Base 
station  rod  rested  on  top  of  the  stake  and  pressed  firmly  against  the  other.  This 
arrangement  ensured  that  the  GPS  system  was  properly  aligned  above  the  same  point 
every  time  it  was  used.  The  bubble  level  on  Base  station  rod  was  used  to  ensure  the 
rod  was  vertical,  and  the  stakes  ensured  the  horizontal  and  vertical  positions  were 
correct.  This  ad  hoc  monument  proved  very  effective  for  positioning  the  base.  Over 
several  days  we  were  able  to  consistently  find  staked  out  points  within  centimeter 
accuracy.  For  future  surveying  efforts,  a  more  permanent  monument  is  suggested. 
Though  the  stakes  were  effective,  after  a  week  of  use,  they  were  beginning  to  loosen, 
and  may  have  become  unstable  with  continued  use.  The  position  of  the  monument  was 
selected  to  provide  the  best  line  of  sight  to  all  of  the  testing  areas.  It  was  on  a  slightly 
elevated  area  that  was  close  to  the  center  of  the  test  ranges. 


Figure  A-7:  Picture  of  Ad-Hoc  Monument. 


Base  Setup 

Our  Promark  3  RTK  Base  station  was  setup  with  two  main  reference  files  “BSUBR”  and 
“BSUBP”  in  the  Fast  Survey  Program.  These  reference  files  serve  to  accurately  relocate 
the  GPS  Base  station  with  respect  to  the  UTM  system  such  that  we  may  have  +/- 1 
meter  accuracy  with  respect  to  the  world  but,  with  respect  to  the  Rover  Stations  location 
relative  to  the  Base  station,  we  have  centimeter  accuracy. 

We  took  999  Points  for  each  of  the  reference  files  to  localize  the  GPS  with  respect  to  the 
world’s  UTM  coordinate  system  using  WGS84  zone  13N.  Our  GPS  elevation  mask  was 
set  at  7  degrees  above  horizontal  based  on  terrain  around  the  Base  station.  We  used 
the  BSUBR  file  for  all  fields  except  the  Subsurface  Practice  field  where  BSUBP  was 
used. 

Physical  setup  was  as  prescribed  by  our  Geonav  Vendor  with  kinematic  bar  type  setup. 
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Figure  A-8:  Physical  Setup  for  Base  Station. 


Rover  Setup 

For  Our  Promark  3  RTK  station  we  used  different  files  for  each  range’s  points  but  had 
similar  settings  as  the  Base  station:  WGS84  zone  13N,  7  degree  elevation  mask.  Our 
RTK  initialization  was  done  as  “on  the  fly”  initialization  after  learning  that  the  kinematic 
bar  did  not  any  more  reliable  results.  We  transcribed  by  hand  all  corner  points  in  UTM 
from  a  Excel  file  provided  to  us  into  each  of  the  field  files.  In  addition  we  created  an 
Outer  Extents  file  which  contained  the  extents  of  the  non-vegetation  removal  fields. 

Physical  setup  was  as  prescribed  by  our  Geonav  Vendor  with  kinematic  bar  type  setup. 


Figure  A-9:  Physical  Rover  Setup. 
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Field  Setup 

Corner  Posts  Placement 

The  GPS  locations  of  the  corners  of  the  test  ranges  were  provided  to  the  surveying 
team.  These  coordinates  were  manually  entered  into  the  GPS  system  during  system 
prep.  At  the  field,  the  “stakeout  point”  function  was  used  to  locate  the  corner  points. 

The  Rover  was  moved  to  the  corner  post  locations,  and  positioned  roughly  on  the  bipod. 
Then  the  Rover’s  rod  was  plumbed  using  the  attached  bubble  level.  Fine  adjustments 
were  made  to  the  position  to  get  the  Rover  within  0.02  meters  of  the  stakeout  point  in 
both  Northing  and  Easting.  A  point  was  logged  at  each  stakeout  position.  Several 
methods  were  tried  to  mark  the  corner  posts,  but  the  most  effective  method  was  to  drive 
a  6”  plastic  stake  into  the  ground  to  mark  the  exact  position  of  the  point  indicated  by  the 
Rover.  This  stake  was  pounded  in  until  it  was  flush  with  the  ground  so  it  would  not  be 
knocked  over  by  ground  operations.  Then  a  3  foot  wooden  stake  was  driven  in  next  to 
the  plastic  stake  to  make  the  spot  easier  to  identify  visually.  Camera  posts  were 
positioned  3  meters  from  the  position  of  the  marked  corner  positions. 


Simulated  UXO  Placement 

Subsurface: 

The  geophysical  mapping  and  subsurface  clearance  test  areas  had  numerous  pieces  of 
simulated  unexploded  ordinance  (SUXO)  buried  at  random  locations  around  the  field. 
These  simulated  pieces  were  14”,  3”  and  6”  diameter  steel  pipe  sections  each  marked 
with  a  2  digit  identifier  number.  These  pieces  were  buried  either  using  shovels  or  a 
small  excavator.  One  team  dug  the  holes  and  placed  the  SUXO’s  in  the  holes.  The 
pieces  were  placed  at  roughly  45  degrees  from  the  vertical  and  pointed  at  various 
orientations  from  north.  The  surveying  team  then  placed  the  tip  of  the  GPS  rod  directly 
on  top  of  the  SUXO  and  plumbed  the  rod  using  the  bubble  level.  The  orientation  of  the 
SUXO  was  measured  as  the  angle  between  the  upper  end  of  the  pipe  and  magnetic 
north  as  shown  on  a  compass.  A  data  point  was  logged  on  each  SUXO  and  the  point 
description  was  edited  to  include  the  size,  number,  and  orientation  of  the  piece.  After 
logging,  the  SUXOs  were  buried.  After  burying  all  of  the  pieces,  the  stakeout  point 
function  was  used  to  locate  each  SUXO  and  a  point  was  again  logged.  The  elevation 
difference  between  the  first  point  and  the  stakeout  point  provide  the  buried  depth  of  the 
SUXO. 


Figure  A-10:  Example  of  Sub  Surface  UXO  Placement  &  GPS  Measurement. 


120 

Distribution  A:  Approved  for  public  release;  distribution  unlimited.  88ABW-2012-5009,  19  Sep  2012. 


Robotic  Ranse 

Clearance  Competition  R2C2  Validation  Trials  AAR  and  Rules  Changes  v.  28  MAY  2011 


Surface: 

The  Surface  clearance  test  areas  had  numerous  pieces  of  simulated  unexploded 
ordinance  (SUXO)  scattered  on  the  surface  of  the  test  range.  These  pieces  included 
sections  of  angle  iron,  !4”  and  3”  diameter  pipe.  These  pieces  were  scattered  manually 
around  the  fields.  No  GPS  locations  were  determined  for  any  of  the  surface  debris. 


Terrain  Mapping 

The  surface  contours  of  the  geophysical  mapping,  surface  clearance,  and  subsurface 
clearance  areas  were  determined  by  logging  a  grid  of  GPS  points  over  the  surface  of  the 
test  areas.  The  areas  were  all  roughly  rectangular.  The  stakeout  line  function  of  the 
GPS  system  was  utilized  to  stakeout  points  along  the  longest  side  of  the  test  range.  The 
corner  points  of  the  test  range  were  used  to  define  this  line,  then  the  Rover  was  walked 
along  this  line  and  points  were  logged  every  10  paces  (roughly  10  meters).  If  significant 
contour  change  (e.g.,  ravines)  happened  within  the  10  meter  spacing,  additional  points 
were  logged  at  the  discretion  of  the  surveyor.  After  logging  all  the  points  along  the  line, 
a  10  meter  offset  was  added  to  the  line  and  the  process  was  repeated.  When  most  of 
the  test  range  had  been  mapped  in  this  way,  the  opposite  side  of  the  range  was  mapped 
(using  the  remaining  two  corner  points).  If  the  sides  of  the  test  range  were  not  parallel, 
additional  points  were  logged  to  ensure  a  10  meter  grid  spacing.  If  the  surface  contour 
will  be  used  to  determine  changes  in  grade,  it  is  a  good  best  practice  to  log  data  points 
outside  of  the  test  range  (offset  1 0  meters  from  the  sides  of  the  test  range).  This  will 
help  to  define  a  smoother  fitted  surface  during  post  processing.  Logging  points  outside 
of  the  test  range  was  only  done  for  the  “subsurface  real”  test  range. 

After  mapping  the  surface  contour,  the  Rover  was  used  to  mark  the  locations  of  notable 
obstacles  in  the  test  range.  For  the  placed  obstacles  (rectangular  stacks  of  railroad 
ties),  GPS  points  were  logged  at  the  4  corners  of  each  stack  at  ground  level.  For  natural 
obstacles  (large  rocks,  trees,  etc.)  4  points  were  logged  at  the  outer  extents  of  the 
obstacles  at  ground  level.  For  every  obstacle,  the  description  of  the  points  was 
modified  to  describe  the  obstacle. 


Q&A 

Geophysical  Mapping 
Disturbance  Documentation  Grid 

For  the  geophysical  mapping  range,  a  grid  was  laid  out  to  help  characterize  the  ground 
disturbance  caused  by  the  competitor.  The  stakeout  line  function  of  the  GPS  system 
was  utilized  to  stakeout  points  along  the  longest  side  of  the  test  range.  The  corner 
points  of  the  test  range  were  used  to  define  this  line,  then  the  Rover  was  walked  along 
this  line  and  points  were  logged  every  20  meters  as  measured  by  a  tape  measure.  The 
process  was  repeated  on  the  opposite  side  of  the  test  range.  Each  logged  point  was 
marked  with  a  3  foot  wooden  stake.  The  staked  lines  were  then  used  to  mark  the  grid. 
The  offset  function  on  the  GPS  was  used  to  align  the  Rover  20  meters  from  the  first  side, 
then  someone  visually  sighted  between  the  two  staked  sides  and  the  Rover  to  align  the 
point.  This  process  was  repeated  to  generate  the  full  grid.  Each  grid  point  was  marked 
with  a  3  foot  wooden  stake. 
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After  marking  the  grid,  the  QA  team  examined  each  grid  area  for  disturbances  caused  by 
the  competitor.  Each  disturbance  feature  that  was  potentially  out  of  spec  was  outlined 
with  bright  orange  spray  paint.  For  depressions  or  holes,  a  photo  was  taken  of  the  hole 
with  a  measuring  tape  stuck  in  the  hole  vertically.  Then  a  GPS  point  was  logged  at  the 
deepest  part  of  the  depression  and  another  was  taken  at  the  nearest  undisturbed  soil. 

For  areas  of  positive  surface  deflection,  a  5  pound  weight  was  dropped  on  the  mound 
from  at  least  1  ft.  above  the  feature.  This  was  repeated  as  necessary  to  push  down 
loose  soil  and  get  an  accurate  measurement  of  the  disturbance.  Two  photographs  were 
taken  after  stamping,  with  a  measuring  tape  in  the  image  for  scaling.  One  photograph 
was  at  roughly  a  45  degree  angle  from  the  ground  and  the  other  was  from  ground  level. 
Then  a  GPS  point  was  logged  on  the  highest  part  of  the  disturbance  and  a  second  point 
was  logged  at  the  nearest  area  of  undisturbed  soil.  This  process  was  to  log  the  exact 
position  and  elevation  change  for  each  point  of  interest. 


Subsurface  Clearance 

Disturbance  of  Dug  Holes 

For  the  subsurface  real  test  range,  QA  was  performed  on  holes  left  behind  from  digging 
up  the  SUXOs.  Photos  were  taken  of  the  hole  and  the  mound  of  dirt  created  from 
digging.  A  tape  measure  was  included  in  each  photo  for  scaling.  Three  GPS  points 
were  logged  for  each  hole.  One  point  was  logged  at  the  deepest  part  of  the  hole.  The 
rough  dimensions  of  the  hole  as  measured  with  the  measuring  tape  were  stored  in  the 
description  of  the  point  using  the  Rover.  The  second  point  was  logged  from  the  highest 
point  of  the  mound  of  dirt  left  from  digging.  The  rough  dimensions  of  the  mount  as 
measured  with  the  measuring  tape  were  stored  in  the  description  of  the  point  using  the 
Rover.  The  final  point  was  measured  from  the  nearest  section  of  undisturbed  ground  to 
give  a  baseline  to  compare  the  disturbance  height. 

R2C2  Competition 

Field  Manual 


There  are  several  intricacies  about  our  GPS  systems  that  were  learned  during  the 
practice  competition.  These  intricacies  are  documented  here.  This  Field  manual  will 
help  the  user  to  conduct  all  GPS  operations  relevant  to  the  R2C2  competition.  Based  on 
what  the  user  needs  to  accomplish  they  can  reference  this  manual  for  exact  inputs  and 
outputs  needed  to  conduct  GPS  operations  fast,  reliably  and  accurately.  Most 
importantly  there  is  a  nomenclature  section  that  details  out  how  all  files,  points,  and 
reference  stations  should  be  labeled. 


Survey  Equipment  Setup 
Monument  Placement  &  Construction 


Monuments  are  used  to  mark  the  position  of  each  Base  station’s  physical  location  and 
thus,  for  each  field  created,  a  monument  must  be  placed.  It  is  important  that  monuments 
are  permanent  and  immovable  as  they  will  be  constantly  used  to  accurately  obtain 
reliable  GPS  positions  of  field  extents,  UXO  positioning,  and  Q&A  procedures.  In  the 
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Field  Manual  we  will  document  a  procedure  which  will  assist  in  optimal  monument 
placement  as  well  as  dictate  guidelines  for  pouring  a  monument. 


Figure  A-11:  Example  Picture  of  a  Surveying  Monument  (From  Google  Image 

Search.) 


Physical  Setup  of  GPS  Equipment 

Base  Station  Setup 

In  the  Field  Manual  we  will  walk  though  how  to  setup  the  Base  Station  the  first  time  on 
unstaked  fields.  It  is  important  to  note  that  the  Base  station’s  position  accuracy  will  be 
about  +/-  1  meter  with  respect  to  the  world.  Competitors  will  have  to  shift  coordinate 
systems  of  boundaries  or  dig  sites  to  be  accurate  in  their  coordinate  space. 


Rover  Station  Setup 

In  the  Field  Manual  we  will  walk  though  a  standard  Rover  Station  setup  assuming  that 
the  files  have  been  created.  The  Setup  described  in  this  section  of  the  Field  Manual  will 
allow  the  user  to  take  new  data  points  or  stake  out  previously  entered  data  points. 
Positioning  of  Rover  is  not  important;  this  set  up  can  be  done  while  Rover  station  is  in 
motion.  For  the  ARC  competition  will  be  running  multiple  Rovers  to  maximize  efficiency 
for  GPS  Surveying  Operations. 


File  &  Nomenclature  System  setup 

We  will  perform  as  much  Setup  before  the  competition  as  is  possible.  This  will 
encompass  creating  all  GPS  files,  excel  tools,  and  data  export  formats  ahead  of  time  to 
maximize  competition  efficiency.  As  part  of  this  effort  we  will  construct  a  detailed 
Nomenclature  section  that  dictates  out  every  file,  surveying  point,  and  Q&A 
documentation  procedure  as  a  main  section  in  the  Field  Manual.  By  setting  up  virtually 
all  files  trees  and  nomenclature  guidelines  ahead  of  time  our  team  can  cross  check 
everything  ahead  of  time  and  ensure  smooth  competition  operations. 
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Field  Setup 

General  Field  Setup 

All  of  the  fields  will  have  “soft”  corner  markings  such  that  if  a  competitors  robot  runs  over 
the  corner  locations  neither  the  corner  nor  the  robot  will  be  damaged.  To  solve  this 
problem  we  will  implement  two  types  of  corner  markers;  high-visibility  markers  and  high- 
accuracy  markers.  High-visibility  markers  will  be  constructed  of  disposable  material  and 
will  be  destroyed  without  harm  to  the  competitor  should  impact  occur.  The  high 
accuracy  markers  will  be  sub  surface  magnetic  nails  such  that  competitor  proposed 
concept  of  operations  should  not  interact  with  this  marker.  Should  impact  of  the  high 
visibility  marker  occur  our  team  will  replace  the  marker  quickly  by  localizing  with  the 
high-accuracy  marker.  The  high-accuracy  marker  will  also  be  available  to  competitors 
should  they  need  to  shift  their  reference  planes. 


Geophysical  Mapping  Field  Setup 

This  range  type  contains  buried  UXOs,  and  obstacles  that  competitors  must  avoid.  For 
the  purposes  of  surveying  operations  the  subsurface  UXOs  need  to  be  GPS  marked  for 
scoring  validation  and  later  removal.  Obstacles  will  have  corner  points  GPS  marked  and 
delivered  to  the  competitor.  Additionally,  a  contour  map  will  need  to  be  taken  before  and 
after  robotic  testing  to  ensure  heavy  equipment  does  not  disturb  the  overall  grade  of  the 
land.  The  Field  Manual  will  illustrate  all  operations  necessary  to  create  this  field  as  well 
as  document  the  correct  export  procedure  to  deliver  competitor  information  (field  corner 
locations,  obstacle  corner  points,  etc.) 


Surface  Clearance  Field  Setup 

This  range  type  contains  surface  UXOs  for  competitor  retrieval  and  obstacles  that 
competitors  must  avoid.  Surface  UXOs  will  be  scattered  randomly  and  their  positions 
will  not  be  noted.  Just  as  in  Geophysical  Mapping,  obstacles  will  have  corner  points 
GPS  marked  and  delivered  to  the  competitor.  Again,  a  contour  map  will  need  to  be 
taken  before  and  after  robotic  testing  to  ensure  heavy  equipment  does  not  disturb  the 
overall  grade  of  the  land.  The  Field  Manual  will  illustrate  all  operations  necessary  to 
create  this  field  as  well  as  document  the  correct  export  procedure  to  deliver  competitor 
information  (field  corner  locations,  obstacle  corner  points,  etc.) 


Sub  Surface  Clearance  Field  Setup 

This  range  solely  has  buried  UXOs  for  customer  retrieval.  These  Buried  UXO  positions 
will  be  GPS  marked  prior  to  and  after  burial.  The  competitor  will  be  provided  the  surface 
GPS  location.  The  subsurface  location  will  be  used  for  oversight  team  retrieval  after  the 
competition  should  they  not  be  recovered  by  the  competitor.  The  surface  locations  of  the 
UXOs  will  also  serve  as  a  benchmark  for  Q&A  criteria  for  disturbance  scoring.  The  Field 
Manual  will  illustrate  all  operations  necessary  to  create  this  field  as  well  as  document  the 
correct  export  procedure  to  deliver  competitor  information  (field  corner  locations,  UXO 
surface  locations,  etc.) 
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Vegetation  Removal  Field  Setup 

This  range  type  has  no  added  features  but  contains  vegetation  to  be  removed.  Large 
vegetation  will  have  its  position  documented  via  GPS  surveying  by  mapping  opposite 
sides  of  the  tree  (or  other)  and  interpolating  a  middle  position  which  will  be  given  to  the 
competitors.  Additionally,  the  oversight  team  will  contour  map  the  entire  area  for  later 
Q&A  evaluation  as  it  pertains  to  debris  and  land  grade  variations.  The  Field  Manual  will 
illustrate  all  operations  necessary  to  create  this  field  as  well  as  document  the  correct 
export  procedure  to  deliver  competitor  information  (field  corner  locations,  large 
vegetation  center  points,  etc.) 


Q&A 

Surface  Damage  Documentation 

All  ranges  will  be  evaluated  based  on  the  latest  version  of  the  Rules  and  Metrics 
Document  as  it  pertains  to  surface  damage.  The  oversight  team  will  place  stakes  in  a 
grid  pattern  (grid  density  TBD  based  on  specific  range  size)  for  damage  evaluation. 
Q&A  team  will  walk  the  grid  and  mark  the  largest  suspect  areas  of  surface  damage  in 
each  grid  unit.  We  will  then  take  scaled  photographs  and  measurements  to  obtain  the 
degree  of  surface  or  subsurface  deflection.  After  initial  measurements  the  Q&A  team 
will  drop  a  5-1 01b  (1ft.  x  1ft.)  object  from  a  height  of  approximately  two  feet  above  all 
positive  surface  deflections.  After  the  weight  drop  we  will  document  all  positive  surface 
deflections.  Negative  surface  deflections  will  be  measured  by  placing  an  appropriately 
sized  disk  in  the  deepest  part  of  the  hole  and  measure  negative  surface  deflection  from 
the  top  of  that  disk.  Specialty  tools/jigs  may  be  created  for  the  purpose  of  standardizing 
deflection  measurements. 


Grade  Change  Documentation 

For  Surface  Clearance  and  Vegetation  Removal  Ranges,  grade  changes  must  be 
documented.  This  documentation  is  performed  by  a  before  and  after  contour  mapping  of 
the  topography.  Contour  mapping  will  be  performed  by  establishing  a  major  grid  spacing 
based  on  area  size  to  be  mapped.  GPS  points  will  be  taken  approximately  along  this 
grid  (+/- 1 .5  meters).  Where  there  is  a  high  degree  of  vertical  deflection  or  natural 
obstacles  the  grid  will  be  refined  appropriately  to  obtain  topography  characteristics.  The 
before  and  after  contour  maps  will  be  compared  based  via  interpolated  mesh  projections 
and  evaluated  based  on  the  Rules  and  Metrics  Document. 


Pile  Cleanliness  Documentation 

For  Surface  and  Subsurface  Clearance  Ranges,  a  pile  of  UXOs  will  be  created.  Each 
pile’s  state  must  be  documented  based  on  the  Rules  and  Metrics  Document. 
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Document  Change  Summary 


Section 

Description 

Date 

Introduction  -  Determination 
of  Cash  Prize  Competition 
Winner 

Removed  minimum  performance  requirement 
definition  deadline. 

13  JAN  2011 

Introduction  -  Minimum 
Performance  Requirements 
for  Cash  Prize  Eligibility 

Minimum  Performance  Requirements  for  Cash 

Prize  Eligibility  section  added 

13  JAN  2011 

Introduction  -  Schedule 

Changed  Competitor  In-Process  Review  schedule 
to  “first  two  weeks  of  April  201 1 .” 

13  JAN  2011 

Rules  -  Operational 

Activities 

Removed  Exclusion  Zone  section  and  replaced 
with  Standoff  Capability  Requirements 

13  JAN  2011 

Metrics  -  Metrics 

Categories 

Added  additional  explanation  of  how  points  in 
metrics  categories  combine  to  create  an  event 
score 

13  JAN  2011 

Metrics  -  System  Task 
Performance  Metrics  - 
Vegetation  Clearance  - 
Small  Vegetation  and 
Vegetation  Residue 

Removal 

Added  “(Remaining  trees  will  also  count  as 
remaining  vegetation.)”  to  metric 

13  JAN  2011 

Metrics  -  System  Task 
Performance  Metrics  - 
Surface  Clearance  - 
Surface  Debris  Removal 

In  A.  c.  (percent  removed  score),  changed  “weight” 
to  “number”. 

13  JAN  2011 

Introduction  -  Minimum 
Performance  Requirements 
for  Cash  Prize  Eligibility 

Changed  language  to  read,  “the  team  must 
participate  in  each  event” 

2  MAR  2011 

Various 

Changed  language  to  consistently  refer  to  “seeded 
metallic  items.” 

12  MAY  2011 

Introduction  -  Rules 
Modifications  and 

Adjudication 

Changed  competition  schedule  to  reflect  IPRs  in 
April/May  2011  and  actual  competition  from  1 

August  to  20  August  201 1 . 

12  MAY  2011 

Introduction  -  Rules 
Modifications  and 

Adjudication 

Added  “Adjudication”  to  “Rules  Modifications” 
section  header. 

12  MAY  2011 

Introduction  -  Rules 
Modifications  and 

Adjudication 

Added  statement,  “The  decisions  and 
determinations  of  the  R2C2  management  and 
judges  are  final.” 

12  MAY  2011 

Metrics  -  System  Task 
Performance  Metrics  -  Sub¬ 
surface  Clearance  -  Sub¬ 
surface  Debris  Removal 

Added  clarification  to  Sub-surface  debris  removal 
metrics  section  regarding  requirement  for  visual 
identification  of  seeded  metallic  items  in  collection 
area  with  unaided  eye. 

12  MAY  2011 

Metrics  -  Man  Hours  to 
Perform  the  Task 

Added  clarification  language  regarding  the 
determination  of  a  man  hours  baseline. 

12  MAY  2011 

Metrics  -  Geophysical 
Mapping  -  Sensor 

Coverage 

Removed  coverage  penalty  for  less  than  95% 
coverage. 

6  AUG  2011 
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Introduction 

Purpose 

The  purpose  of  the  Robotic  Range  Clearance  Competition  (R2C2)  is  to  quickly  tap  into 
the  innovation  and  ingenuity  of  the  commercial  robotic  technology  sector  to  improve  the 
safety  and  effectiveness  of  any  or  all  of  the  four  tasks  traditionally  associated  with  range 
clearing  operations: 


1 .  Vegetation  removal 

2.  Surface  clearance 


3.  Geophysical  mapping 

4.  Sub-surface  clearance 


Background 

Range  clearance  operations  as  currently  conducted  are  manpower  intensive,  time 
consuming,  dangerous,  and  expensive.  Initial  data  from  robotic  range  clearance 
technology  development  efforts  indicate  the  strong  potential  for  significant  reductions  in 
the  time  and  cost  required  to  conduct  range  clearing  operations.  Experiments  to  date 
indicate  the  possibility  of  reducing  range  clearance  times  by  two  thirds  and  costs  by  one 
third  if  automated  clearing  equipment  is  used. 

Additionally,  there  are  no  automated  “commercial  off  the  shelf”  solutions  available  for  the 
Department  to  procure.  The  traditional  approach  of  establishing  an  R&D  program  that 
can  be  transitioned  into  a  development  and  acquisition  program  is  not  possible  within  the 
desired  quick  turn-around  timeframe  as  none  of  the  uniformed  services  have 
programmed  funding  for  this  traditional  approach.  By  exercising  statutory  authority  under 
Section  2374a  of  title  10  United  States  Code  as  amended  by  Section  212  of  the  John 
Warner  National  Defense  Authorization  Act  for  Fiscal  Year  2007  Public  Law  109-364  to 
offer  a  cash  prize  for  development  robotic  range  clearing  application,  it  may  be  possible 
to  provide  the  desired  capability  in  a  significantly  shorter  amount  of  time. 

Scope 

The  tasks  associated  with  range  clearance  that  likely  have  the  greatest  potential  for 
applying  ground  robotics  technology  include  automated  vegetation  clearance,  automated 
surface  debris  clearance,  automated  geophysical  mapping,  and  automated  Sub-surface 
anomaly  excavation.  The  competition  will  assess  the  ability  of  competitor  systems  to 
provide  increased  safety  and  operational  effectiveness  to  range  clearance  operations. 
The  competitor  systems  are  expected  to  have  applied  robotics  technology  to  all  or  some 
appropriate  combination  of  the  inherent  range  clearance  tasks  in  a  range  clearance 
operation.  Because  the  competition  is  focused  on  increasing  safety  and  operational 
effectiveness  via  robotics  automation  as  well  as  reducing  time  and  cost,  competitors  are 
not  expected  to  attempt  to  develop  improved  vegetation  removal  tools  or  geophysical 
detection  and  identification  sensor  technology. 
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Objective 

The  objective  of  this  competition  is  to  advance  robotic  technology  used  in  range 
clearance  operations  in  order  to  increase  operational  effectiveness  while  providing 
greater  safety  for  range  clearance  team  members. 

General  Competition  Objectives 

Operations  should  be  as  automated  as  practical.  Full  automation  is  desired  but  not 
required  to  compete  in  the  competition.  Unmanned  tele-operation  is  the  minimum 
acceptable  system  requirement  to  compete  in  the  competition. 

No  downrange  manual  operations  are  authorized  with  the  sole  exception  of  deliberate 
system  recovery  operations. 

The  pre-existing  grade  shall  not  be  changed  more  than  +/- 15  cm. 

Leave  each  site  with  no  surface  damage  that  would  prevent  a  light  truck  from  traversing 
the  site. 

In  addition,  systems  will  demonstrate  the  following  category  objectives: 

Vegetation  Removal  Objectives 

The  competition  is  intended  to  demonstrate  the  following  Vegetation  Removal 
Objectives: 

A.  Remove  all  vegetation  to  a  height  no  greater  than  8  cm  above  surrounding 
grade.  This  will  include  trees  up  to  36  cm  in  diameter  (measured  at  122  cm 
height). 

B.  Vegetation  residue  must  be  mulched  or  removed  from  the  site.  Vegetation 
residue  left  on  the  site  must  be  mulched  to  a  height  not  greater  than  15  cm  in 
depth.  Residue  removed  from  the  site  must  be  placed  in  a  designated  area  within 
300  meters  of  the  site. 

Surface  Clearance  Objectives 

The  competition  is  intended  to  demonstrate  the  following  Surface  Clearance  Objectives: 

A.  Remove  all  seeded  metallic  items  from  the  surface  of  the  contest  site  in  the 
areas  designated. 

B.  The  site  will  be  seeded  with  metallic  items  20  mm  in  width  or  greater. 

C.  All  seeded  metallic  items  removed  from  the  clearance  site  shall  be  placed  in  a 
designated  collection  area  within  300  meters  of  the  site. 

Geophysical  Mapping  Objectives 

The  competition  is  intended  to  demonstrate  the  following  Geophysical  Mapping 
Objectives: 

A.  Navigate  a  digital  geophysical  mapping  platform  within  the  designated  area  to 
collect  digital  geophysical  data  so  that  seeded  metallic  objects  can  be  detected 
and  located  to  30  cm  or  better  positional  accuracy. 

B.  Collect  raw  geophysical  data  with  an  objective  noise  level  to  be  determined  at  the 
site. 

C.  Deploy  a  time  domain  electromagnetic  induction  metal  detector  and  record  its 
data  over  100%  of  the  designated  area  at  a  line  spacing  of  50  cm. 
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The  geophysical  mapping  platform  shall  be  operated  at  a  consistent  speed  to  be 
determined  by  the  competitor. 

Sub-surface  Clearance  Objectives 

The  competition  is  intended  to  demonstrate  the  following  Sub-surface  Clearance 
Objectives: 

A.  Remove  all  seeded  metallic  items  identified  in  a  supplied  dig  list  from  the 
designated  area  in  the  contest  site.  Seeded  metallic  item  depths  shall  not  exceed 
1  meter. 

B.  The  site  will  be  seeded  with  metallic  items  60  mm  in  width  or  greater. 

C.  All  seeded  metallic  items  removed  from  the  clearance  site  shall  be  placed  in  a 
designated  sub-surface  clearance  collection  area  within  300  meters  of  the  site. 

Competition  Metrics 

The  metrics  for  the  competition  will  be  derived  from  the  following  areas: 

A.  System  task  performance 

B.  Level  of  human  interaction 

C.  Man-hours  to  perform  the  task 

Determination  of  Cash  Prize  Competition  Winner 

Competitors  may  choose  to  participate  in  all  or  some  smaller  combination  of  the  4 
Events:  (1)  Vegetation  Removal,  (2)  Surface  Clearance,  (3)  Geophysical  Mapping,  and 
(4)  Sub-surface  Clearance. 

The  Ultimate  Cash  Prize  will  be  awarded  based  on  best  performance  against  metrics, 
not  one  system  against  another.  Competitors  will  score  points  based  on  their  system’s 
performance  within  each  of  the  Events.  The  competitor  that  performs  in  all  four 
categories  and  receives  the  highest  score  across  all  of  the  events  will  be  declared  the 
Overall  Competition  Winner.  Additional  cash  prizes  will  be  awarded  to  the  best 
performance  in  each  of  the  four  events. 

A  Cash  Prize  of  $1 M  will  be  awarded  to  the  Overall  Competition  Winner  having  met 
competition  objectives  and  participated  in  all  four  events.  A  Cash  Prize  of  $250K  will  be 
awarded  to  the  best  performance  having  met  competition  objectives  in  each  of  the 
events.  It  will  be  possible  for  the  Overall  Winner  of  the  $1 M  cash  prize  to  also  win  the 
event  cash  prizes.  The  possible  cash  awards  can  range  from  one  competitor  winning  the 
entire  $2M  (overall  and  best  in  each  of  the  four  events)  to  one  competitor  winning  the 
Overall  Cash  Prize  and  four  different  competitors  each  winning  a  $250K  Event  Cash 
Prize.  Therefore  anywhere  from  one  to  five  competitors  could  finish  the  competition  with 
a  cash  prize,  it  is  possible  that  no  cash  prize  is  awarded  in  one  or  more  of  the 
competition  events  if  system  performance  does  not  meet  the  minimum  requirements. 

Minimum  Performance  Requirements  for  Cash  Prize  Eligibility 

For  each  event  a  minimum  score  of  60  of  250  points  must  be  achieved  in  the  system 
task  performance  category  and  a  minimum  overall  event  score  of  100  of  250  points  must 
be  achieved  in  order  to  be  eligible  for  the  event  cash  prize.  For  a  team  to  be  eligible  for 
the  Ultimate  Cash  Prize  the  team  must  participate  in  each  event  and  have  a  minimum 
combined  overall  score  of  600  of  1000  total  possible  points. 
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Rules  Modifications  and  Adjudication 

Robotic  technology  advancement  is  a  key  objective  of  the  R2C2.  Competitors  are  invited 
to  communicate  directly  with  R2C2  regarding  any  rule  that  restricts  their  ability  to 
demonstrate  technical  achievement  and  innovative  solutions. 

The  R2C2  has  the  authority  to  modify  the  rules  at  any  time.  Rules  may  be  modified  for 
many  reasons,  including  accommodation  of  a  promising  technical  approach  that  would 
have  been  prohibited  by  the  rules. 

R2C2  will  communicate  any  modifications  to  the  rules  through  the  competitor  SharePoint 
site  and  the  Competition  website. 

Clarification  questions  fielded  by  the  R2C2  shall  be  answered  in  the  Competition  Rules 
Frequently  Asked  Questions  (FAQ).  Questions  that  have  been  received  but  not  yet 
answered  will  also  be  posted  in  the  FAQ. 

The  R2C2  may  revise  the  schedule  at  any  time  and  interpret  the  rules  in  any  manner  to 
best  meet  R2C2’s  objectives.  The  decisions  and  determinations  of  the  R2C2 
management  and  judges  are  final.  The  R2C2’s  decisions  are  based  on  a  number  of 
factors  such  as  fairness,  safety,  statutes,  program  goals,  environmental  protection,  and 
efficient  operations. 


Schedule 

Below  is  a  notional  schedule  of  competition  activities. 


Activity 

Location 

Date 

Kick  Off  Event 

Crystal  City 

22  October  2009 

Industry  Day 

Tyndall  AFB 

10  Dec  2009 

Signed  Letters  of  Intent 

Online 

3  May  2010 

Category  Registration 

Online 

3  May  2010 

Optional  Competitor 

Testing 

Camp  Guernsey 

1  Aug  to  1  Nov  2010  and  1 
May  to  1  July  201 1 

Competitor  In-Process 
Reviews 

Competitor  site 

April/May  2011 

Prize  Competition  Packets 
Issued 

Online 

90  days  prior  to  Prize 
Competition 

Prize  Competition  Packets 
Due 

Online 

45  days  prior  to  Prize 
Competition 

Prize  Competition 

Camp  Guernsey 

1  August  through  20  August 
201 1 
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Eligibility  &  Participation 

Participation  Process 

In  order  to  participate  and  be  prize  eligible,  interested  parties  must  do  the  following: 

1 .  Form  a  team  (team  is  defined  as  one  or  more  competitors) 

a.  Team  leader  signs  a  Letter  of  Intent 

b.  Receive  access  to  team  SharePoint 

2.  Choose  competition  categories 

3.  In-process  Review  (IPR) 

a.  Host  IPR  meeting 

4.  Compete 

a.  Submit  Prize  Competition  Packet 

b.  Participate  in  Prize  Competition 

Team  Requirements 

The  R2C2  is  a  competition  between  competitor  teams.  Competitors  will  form  teams  on 
their  own  and  said  teams  will  meet  the  following  requirements. 

Team  Leader 

Competitor  teams  must  have  a  team  leader.  The  team  leader  must  be  an  individual  and 
must  sign  the  Letter  of  Intent. 

Team  Members 

Individuals  may  only  be  on  one  team. 

Vendors 

There  are  no  vendor  restrictions.  Government  Furnished  Equipment  (GFE)  is  limited  to 
R2C2  approval. 

Team  Name 

Teams  must  be  given  a  name.  R2C2  reserves  the  right  to  disapprove  names.  Names  will 
be  granted  on  a  first  come  first  served  basis. 

Letter  of  Intent 

Letters  of  Intent  (LOI)  will  be  posted  at  www.roboticranqeclearance.com  when  available. 
The  team  lead  for  each  competitor  team  must  complete  and  turn  in  the  LOI.  The  LOI 
establishes  who  the  competitor  teams  are.  The  LOI  is  not  a  binding  agreement  that 
compels  participation  in  the  R2C2.  Teams  may  withdraw  from  the  competition  at  any 
time,  in  writing,  without  penalty. 
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Competitor  SharePoint 

Once  a  team’s  LOI  is  received  a  SharePoint  account  will  be  created.  The  SharePoint  is 
the  primary  communication  clearinghouse  for  the  competition.  Up  to  two  individuals  may 
be  given  access  per  team.  The  Competitor  SharePoint  is  not  intended  for  use  as  an 
internal  team  tool  but  for  external  team  interaction  with  the  competition  oversight. 

The  Competitor  SharePoint  has  two  parts: 

-  An  all  competitor  level  that  includes  competition  documents,  the 
competition  calendar,  and  discussion  board. 

-  A  team  folder  that  can  only  be  accessed  by  your  team  and  the 
competition  oversight. 

Category  Registration 

Teams  may  choose  to  participate  in  all  or  some  smaller  combination  of  the  4 
performance  categories:  (1)  Vegetation  Removal,  (2)  Surface  Clearance,  (3) 

Geophysical  Mapping,  and  (4)  Sub-surface  Clearance.  Once  teams  are  given  access  to 
the  SharePoint  site  they  must  inform  R2C2  of  the  categories  the  team  will  be 
participating  in. 

This  will  allow  R2C2  adequate  time  to  resource,  approve,  and  build  the  surrogate  ranges 
for  the  prize  competition. 

Optional  Competitor  Testing 

Any  team  that  does  not  have  access  to  adequate  developmental  test  facilities  may  utilize 
Camp  Guernsey  with  prior  approval.  Use  of  Camp  Guernsey  for  optional  competitor 
team  testing  will  have  no  bearing  on  the  outcome  of  the  prize  competition.  Packets  will 
be  posted  on  the  SharePoint  with  all  required  information  due  45  days  before  your 
requested  test  period.  Packets  contain  forms  that  must  be  filled  out  and  returned  to 
Camp  Guernsey  that  will  allow  the  oversight  team  to  appropriately  resource  and 
schedule  the  testing.  Testing  space  will  be  available  from  1  Aug  to  1  Nov  2010  and  1 
May  to  1  July  201 1 .  No  billeting  is  available  in  this  period.  A  list  of  available  Camp 
Guernsey  resources  will  be  posted  on  the  SharePoint. 

In-Process  Review 

Participation  is  required  for  entrance  to  the  prize  competition. 

The  team  shall  present  a  technical  briefing  to  the  R2C2  Competition  representatives  who 
will  review  technical  approach,  progress,  schedule,  and  risks  at  a  time  to  be  scheduled 
between  the  R2C2  Oversight  Team  and  each  Competitor  Team. 
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Rules 

The  rules  are  intended  to  promote  the  widest  variety  of  technical  solutions. 

The  competition  is  structured  to  represent  active  range  clearance  prior  to  new  range 
construction  on  an  Army  facility. 

Safety  is  the  number  one  priority. 

Operational  Safety 

During  operation,  no  personnel  shall  be  closer  to  a  moving  Unmanned  Ground  Vehicle 
(UGV)  than  the  operator  and  personnel  will  comply  with  their  team’s  approved  safety 
plans  at  all  times. 

Safety  Officer 

Teams  shall  designate  a  safety  officer  for  the  duration  of  the  competition.  Safety  shall  be 
his/her  sole  responsibility  while  the  UGV  is  operating.  UXO  technicians  are  not  required. 

Emergency  Stop  (E-Stop) 

The  system  shall  have  an  Emergency  Stop  (E-Stop).  The  system  must  halt  within  15 
meters  and  cease  all  equipment  operations  when  the  E-Stop  is  initiated. 

Warning  Devices 

Each  vehicle  shall  be  equipped  with  a  warning  light  that  is  activated  according  to  the 
state  of  the  E-Stop  system. 

Each  vehicle  shall  display  one  or  more  flashing  amber  warning  lights,  the  combination  of 
which  results  in  visibility  360  degrees  azimuthally  around  the  vehicle.  The  warning 
light(s)  shall  operate  when  the  vehicle  is  in  E-stop  RUN  mode.  The  vehicle  may  not 
commence  movement  until  the  warning  light(s)  has  been  in  operation  for  5  seconds.  The 
warning  light(s)  shall  comply  with  SAE  Class  1  standards  for  warning  lights  and  shall  not 
produce  light(s)  than  can  be  confused  with  those  of  public  safety  vehicles  such  as  law 
enforcement,  fire,  or  ambulance. 

Loss  of  Communications  Stop 

The  system  shall  automatically  halt  and  cease  operations  if  communications  with  the 
system  are  lost  or  interrupted  for  a  maximum  of  2  seconds  and  may  travel  no  farther 
than  30  meters. 

No  Freewheel 

The  systems  shall  not  be  capable  of  motion  when  stopped  or  un-powered. 

For  example,  systems  that  would  roll  downhill  if  shut  off  are  considered  freewheeling  and 
are  un-safe  for  competition. 

Speed  Limit 

The  Camp  Guernsey  North  Training  Area  speed  limit  is  30  MPH.  Systems  must  comply 
with  regulated  speed  limits. 
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Applicable  Safety  Documents 

DA-PAM  385-10  Army  Safety 
EN  385-1-1 

Radiated  Energy  Safety  Standards 

•  Competitors  are  directed  to  OSHA  29  CFR  1926.54  and  OSHA  Technical  Manual 
(TED  1-0. 15A),  Section  III  -  Chapter  6  (1999,  January  20)  for  relevant  laser 
safety  standards. 

•  Competitors  are  directed  to  OSHA  29  CFR  1910.97  (Non-ionizing  Radiation)  and 
Department  of  Defense  Instruction  6055.1 1  (1995,  February  21)  for  relevant  RF 
safety  standards. 

•  Competitors  are  directed  to  OSHA  29  CFR  1910.95  (Occupational  Noise  Control) 
and  OSHA  Technical  Manual  (TED  1-0. 15A),  Section  III  -  Chapter  5  (1999, 
January  20)  for  relevant  acoustic  safety  standards. 

Operational  Activities 

Event  Times 

R2C2  Range  Manager  will  call  the  official  start  and  stop  times  of  each  event.  The  total 
duration  of  each  event  will  be  a  maximum  of  8  hours.  The  start  times  will  be  coordinated 
with  each  team  prior  to  the  individual  event  and  subject  to  change  for  any  management 
or  safety  need  as  deemed  necessary  by  the  R2C2  Oversight  Team. 


The  event  may  be  paused  for  competition  event  officials  and/or  the  competitors  to  take 
sustenance  breaks.  Any  pauses  required  for  sustenance  breaks  will  not  be  counted 
against  the  event  time. 

Standoff  Capability  Requirements 

Teams  must  have  the  capability  to  fully  operate  their  systems  at  a  range  of  870  meters. 
This  is  intended  to  represent  the  surface  danger  zone  of  large  munitions.  In  addition  the 
command  locations  shall  not  have  guaranteed  direct  line  of  sight  to  the  operation  sites. 

No  personnel  on  range 

No  team  personnel  may  enter  the  range  at  any  time  without  R2C2  Management  Team 
and  Range  Safety  Officer  authorization.  Violations  will  result  in  team  member 
disqualification. 

Area  of  Operation  boundary 

No  system  shall  be  more  than  3  meters  outside  the  perimeter  of  the  area  of  operation 
(AO)  or  a  penalty  will  be  assessed.  The  AO  boundaries  will  be  defined  in  a  SHAPE  file 
provided  to  the  competitor  team. 

Mapping  data 

All  raw  geophysical  data  is  to  be  delivered  to  the  R2C2  in  the  raw  data  format  defined  in 
Section  4  of  Data  Item  Description  (DID)  MMRP-09-04. 
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This  DID  can  be  found  at  the  following  web  site  address: 

http://www.hnd.usace.army.mil/oew/policy/dids/FY09_MMRP_DIDS/MMRP-09-004.pdf. 
The  data  shall  be  provided  by  the  competitor  in  two  forms. 

First  the  raw  unfiltered  geophysical  data  shall  be  provided  after  the  conclusion  of  daily 
data  gathering  operations  before  departing  the  range.  Next  the  competitor  shall  provide 
the  raw  data  in  a  format  where  it  has  been  merged  with  the  navigation  data.  The  raw 
geophysical  data  shall  NOT  be  filtered  in  any  manner. 

Second,  positioning  data  may  be  filtered  and  merged  with  positional  data  to  produce 
what  the  competitor  believes  to  be  the  best  representation  of  where  geophysical  data 
measurements  were  actually  recorded.  All  such  filtering  must  be  documented  in  the 
metadata  file(s)  accompanying  the  geophysical  data  delivery.  The  merged  data  must  be 
delivered  to  the  R2C2  Oversight  Team  within  48  hours  of  the  completion  of  the 
geophysical  data  collection. 
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Metrics 

General  Scoring 

Competitors  may  choose  to  participate  in  all  or  some  smaller  combination  of  the  4 
Performance  Areas:  (1 )  Vegetation  Clearance,  (2)  Surface  Debris  Clearance,  (3) 
Geophysical  Mapping,  and  (4)  Sub-surface  Debris  Clearance.  Metrics  and  scoring 
methods  will  be  discussed  for  each  of  the  performance  areas. 

The  overall  philosophy  of  the  scoring  is  that  for  each  event,  teams  will  begin  with  the 
maximum  points  for  each  category  and  will  lose  points  as  solutions  fail  to  achieve  the 
objective  requirements.  Each  event  is  worth  a  maximum  of  250  points  and  will  be 
evaluated  with  three  categories  of  metrics  (system  performance,  level  of  human 
interaction,  and  man-hours  to  perform.)  The  metric  and  scoring  breakdowns  for  each  of 
these  categories  and  how  they  apply  to  each  event  are  described  below. 

Event  and  Competition  Scoring 

Each  event  is  valued  equally  in  the  overall  competition  score,  with  each  event  worth  250 
points.  A  team’s  overall  competition  score  will  be  determined  by  the  sum  of  the  individual 
event  scores  for  a  maximum  total  of  1000  points. 

Scores  in  individual  events  will  be  accumulated  from  evaluation  of  metrics  in  three 
categories. 

Metrics  Categories 

Each  event  has  three  categories  of  metrics. 

A.  System  task  performance 

System  task  performance  metrics  will  address  the  objectives  related  to  system 
performance  specific  to  each  event.  Higher  levels  of  performance  will  result  in  higher 
scores. 

B.  Level  of  human  interaction 

Level  of  human  interaction  metrics  will  address  competition  objectives  related  to 
encouraging  higher  levels  of  autonomy.  Lower  levels  of  human  interaction  will  result  in 
higher  scores  while  higher  levels  of  human  interaction  will  result  in  lower  scores. 

C.  Man  hours  to  perform  the  task 

The  Man-hours  to  perform  the  task  metric  will  address  the  competition  objective  of 
reducing  manpower  required  to  perform  the  tasks.  As  man-hours  to  perform  a  task  or 
event  remain  low,  points  deducted  will  be  low  (resulting  in  a  higher  score).  As  man-hours 
to  perform  a  task  increase,  via  either  increased  personnel  or  increased  task  duration, 
points  deducted  will  increase  (resulting  in  a  lower  score). 

Each  of  these  event  categories  is  worth  250  points,  but  weighted  according  to  the 
breakdown  in  the  next  section  (Metrics  Category  Weights)  so  that  the  event  score 
maximum  is  250  points  total. 

Metrics  Category  Weights 

Each  metric  category  will  have  a  different  weight  for  each  event  score.  The  breakdown  is 
as  follows: 
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A.  System  task  performance  (50%) 

B.  Level  of  human  interaction  (40%) 

C.  Man  hours  to  perform  the  task  (10%) 

System  Task  Performance  is  the  most  important  factor  in  the  evaluation.  Level  of  Human 
Interaction  is  less  important  than  System  Task  Performance.  Man  Hours  to  perform  the 
task  is  less  important  than  Level  of  Human  Interaction.  Metric  weighting  is  consistent 
across  events. 

Penalties  in  Scope  of  the  Entire  Competition 

These  are  penalties  that  are  assessed  at  the  event  level,  but  are  the  same  for  each 
event. 

A.  Teams  will  be  assessed  a  50  point  penalty  each  time  a  robot  leaves  the 
range  perimeter  or  enters  a  no-go  area  on  the  range  except  when  going  to 
the  pit  area  or  disposal  area. 

B.  Team  members  will  be  disqualified  from  the  competition  for  entering  the  870 
meters  exclusion  zone  during  robotic  clearance  operations  without 
authorization  from  the  R2C2  Management  Team  and  Range  Safety  Officer. 

Surface  Damage 

A.  The  site  will  be  divided  into  grid  areas  for  assessment. 

B.  Any  surface  damage  will  be  measured.  Depths  greater  than  15  cm  will  be 
penalized. 

C.  Points  will  be  deducted  for  any  instance  in  a  grid  area. 

D.  Maximum  is  one  penalty  per  grid  area. 

Surface  Grade 

A.  A  survey  of  a  pre-determined  set  of  points  before  and  after  operations  will 
be  conducted  by  R2C2.  The  survey  points  are  notionally  the  corners  of 
the  range,  where  the  gridlines  would  intersect  the  range  boundary  and 
where  the  gridlines  would  intersect  on  the  range  itself. 

B.  Points  will  be  deducted  if  the  absolute  value  of  the  difference  between 
each  measurement  from  the  survey  points  is  more  than  15  cm. 

Tie-breaking  Procedures 

A.  Tie-breaking  procedures  are  only  invoked  to  determine  an  event  or  overall 
winner. 

B.  In  the  event  of  a  points  tie  the  team  with  the  fastest  time  will  be  the  winner. 

C.  In  the  event  of  a  point  and  time  tie  then  the  solution  with  the  greatest 
autonomy  will  be  the  winner. 
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System  Task  Performance  Metrics 
Vegetation  Clearance 

Vegetation  Tree  Removal  (maximum  125  points) 

A.  Scoring  will  be  determined  by  measuring  tree  stumps  on  the  event  range. 
Teams  will  begin  the  event  with  250  points.  Points  will  be  deducted  for 
each  tree  or  stump  remaining  greater  than  8  cm  above  the  surrounding 
grade. 

B.  The  vegetation  removal  score  will  be  difference  between  125  and  the 
deductions. 

Small  Vegetation  and  Vegetation  Residue  Removal  (maximum  125 
points) 

A.  The  site  will  be  divided  into  grid  areas. 

B.  Points  will  be  deducted  for  each  grid  area  where  remaining  vegetation  or 
vegetation  residue  is  greater  than  15  cm  above  the  surrounding  grade. 
(Remaining  trees  will  also  count  as  remaining  vegetation.) 

C.  The  small  vegetation  and  vegetation  residue  removal  score  will  be 
difference  between  125  and  the  deductions. 

Geophysical  Mapping 

Delivery  of  Raw  data 

Raw  data  should  be  provided  to  the  judges  within  30  minutes  of  completing  the 
geophysical  mapping  event.  The  merged  data  should  be  delivered  to  the  R2C2 
Oversight  Team  within  48  hours  of  the  completion  of  the  geophysical  data  collection.  If 
the  team  does  not  deliver  the  merged  data  by  the  deadline,  the  score  for  the  entire 
geophysical  mapping  event  shall  be  zero. 

Noise  Level  (70  Points) 

A.  Using  Oasis  Montaj’s  QC  tools  determine  the  average  level  of  RMS  noise 
in  the  raw  data. 

a.  Baseline  noise  will  be  determined  by  the  R2C2  Competition  Team. 

B .  S core  will  be  determined  by  the  following _ 


Noise  Range 

Points  Awarded 

Low  RMS  noise  level 

Most 

Moderate  RMS  noise  level 

Some 

High  RMS  noise  level 

Least 

C.  Noise  ranges  to  be  determined. 

Sensor  Coverage  (60  points) 

A.  Score  will  be  determined  by  the  percent  of  the  range  covered. 

B.  Score  will  be  determined  by  the  following: 

Percentage  of  coverage  multiplied  by  60  points  =  Score 
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Anomaly  Location  Accuracy  (60  points) 

A.  Determine  anomaly  peak  response  locations  using  Oasis  Montaj's  UX- 
Detect  automatic  anomaly  detection  algorithm  on  the  submitted  merged 
data. 

B.  Number  detected  scoring 

(#  of  Detections  within  30cm  of  actual  burial  location  /  #  seeded  metallic 
items)*  60  =  detection  score. 

Survey  Speed  (60  points) 

A.  Using  Oasis  Montaj,  the  point-to-point  velocities  will  be  calculated  for  all 
data  collected. 

B.  Scoring  will  be  based  on  percentage  of  the  data  collected  at  the  specified 
design  speed. 

a.  Design  speed  will  be  determined  by  competitor  team  prior  to 
event. 


C.  The  scoring  will  be  determined 

uy  the  following  table: 

Design  speed  range 

Points 

98%  to  100%  within  +  or  -  0.3MPH 
of  design  speed: 

60  points 

95%  to  98%  within  +  or  -  0.3MPH 
of  design  speed: 

40  points 

75%  to  95%  within  +  or  -  0.3MPH 
of  design  speed: 

20  points 

50%  to  75%  within  +  or  -  0.3MPH 
of  design  speed: 

5  points 

less  than  50%  within  +  or  -  0.3MPH 
of  design  speed: 

0  points 

Surface  Clearance 

Surface  Debris  Removal  (250  points) 

Site  will  be  seeded  with  material  representing  surface  debris.  The  objective  is  to  remove 
all  seeded  metallic  items  and  to  place  just  the  seeded  metallic  items  removed  from  the 
site  into  a  designated  collection  area. 

A.  Percent  removed  score 

a.  After  surface  clearance  operations  are  complete,  the  site  will 
be  Quality  Control  (QC)  swept  to  determine  remaining  seeded 
metallic  items. 

b.  Percent  removed  score  will  be  calculated  as  follows: 

c.  (Number  of  seeded  metallic  items  in  collection  area  -  Number 
of  seeded  metallic  items  present  in  QC  pickup)/Number  of  total 
seeded  metallic  items  *  125  points. 

B.  Pile  cleanliness  score 

a.  Pile  cleanliness  score  will  be  calculated  as  follows: 

b.  Average  pile  percent  clean  *  125  points,  where  average  pile 
percent  clean  is  determined  by  sampling  the  collection  pile  and 
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determining  the  percent  of  the  pile  that  is  seeded  metallic 
items  as  a  function  of  the  total  material  in  the  samples. 

C.  Surface  debris  removal  score  will  be  the  sum  of  percent  removed 
score  and  pile  cleanliness  score. 

Sub-Surface  Clearance 

Sub-surface  Debris  Removal  (250  points) 

A.  Teams  begin  the  event  with  250  points 

B.  Each  competitor  team  will  be  given  a  set  of  dig  coordinates. 

C.  5  points  will  be  deducted  for  each  seeded  metallic  item  not  recovered 
and  placed  in  the  collection  area. 

D.  Points  will  be  deducted  for  each  seeded  metallic  item  placed  in  the 
collection  area  that  is  not  visually  recognizable  in  its  placed  condition 
by  the  unaided  human  eye. 

E.  Points  will  only  be  deducted  once  for  each  seeded  metallic  item  not 
recovered  or  recognizable  in  the  collection  area. 

F.  No  sub  surface  seeded  metallic  item  intended  for  removal  will  be  at  a 
depth  greater  than  1  meter. 

Level  of  Human  Interaction 

Human  Interaction,  for  the  purpose  of  the  R2C2,  is  defined  as:  any  time  an  operator  or 
team  member  uses  any  human-machine  interface  device  such  as,  but  not  limited  to, 
joystick,  keyboard,  or  voice  recognition  software  to  control  any  machinery  engaged  in  a 
competition  task. 

A.  Teams  will  begin  each  event  with  250  points  in  this  category. 

B.  Competition  team  members  will  be  observed  by  a  R2C2  judge  and  will  be  filmed 
for  reference. 

C.  According  to  the  professional  judgment  of  a  R2C2  robotics  expert  (judge),  the 
Level  of  Human  Interaction  involved  with  the  performance  of  the  task  of  the  total 
system  will  be  determined  according  to  the  following  table: 


Percent  Score 

Description  of  Level  of  Human  Interaction 

100% 

Near  zero  human  interaction 

75% 

Minimal  Interaction 

50% 

Moderate  Interaction 

25% 

Frequent  Interaction 

0% 

Continuous  Interaction  (completely  tele-operated.) 

a.  This  judgment  will  account  for  number  of  operators  and  number  of  robotic 
systems. 

D.  Scores  will  be  computed  by  the  percent  score  *  250  as  a  function  of  the  level  of 
human  interaction  judged  to  have  been  achieved  by  the  expert  judge. 

E.  Complete  tele-operated  solutions  will  get  zero  points  for  this  metric. 
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Man  Hours  to  Perform  the  Task 

A.  Defined  as  Total  time  to  perform  the  task  multiplied  by  the  number  of  people  on 
site. 

a.  The  number  of  people  on  site  will  be  defined  as  the  number  of  people 
actively  involved  in  the  operation. 

b.  Team  observers  or  spectators  will  be  allowed,  but  any 
observers/spectators  will  not  be  allowed  to  interact  with  the  competition 
team  during  the  event. 

B.  The  team  captain  will  notify  the  R2C2  Range  Manager  when  the  team  is  ready  to 
start  the  task.  This  will  count  as  the  event  start  time  for  the  purpose  of  measuring 
the  “Man  Hours  to  Perform  the  Task  Category.” 

C.  End  time  will  be  called  when  the  competitor  team  captain  says  they  have  finished 
(and  when  the  system  has  returned  to  the  pit  area)  or  when  the  time  limit  of  the 
event  is  reached. 

D.  Active  team  members  will  be  designated  by  the  team  prior  to  the  competition 
event  and  each  member  will  receive  identification  (referred  to  as  a  “pit  pass”) 
designating  that  status. 

E.  Any  team  member  without  a  pit  pass  who  is  deemed  by  the  competition  to  be 
engaged  in  assisting  the  team  during  the  event  will  be  added  to  the  team  roster 
and  will  be  included  in  the  man-hours  calculation. 

F.  The  robot  must  be  in  the  pit  area  for  the  end  of  the  exercise  to  be  called  prior  to 
the  expiration  of  the  time  limit. 

G.  Event  durations  will  be  capped. 

H.  “Phone  a  friend”  does  not  count  against  man  hours  metric. 

I.  Safety  officer  does  not  count  against  man  hours. 

J.  Score  will  be  calculated  as  (man  hours  allotted  (baseline)  /  man  hours  required)  * 
250  with  a  250  point  maximum.  The  baseline  will  be  determined  as  the  number  of 
team  members  actively  involved  in  operating  the  system(s)  at  the  start  of  each 
event.  This  will  include  any  spotters,  observers,  technical  advisors,  and  similar 
personnel  determined  by  the  R2C2  to  be  actively  involved  with  operating  the 
system(s). 
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Appendix  D:  Robotic  Range  Clearance  Competition  (R2C2) 

Judges  Guide 


Introduction 


The  purpose  of  this  guide  is  to  help  prepare  the  Robotic  Range  Clearance  Competition  (R2C2) 
judges  and  officials  for  their  duties  at  the  competition  event. 

The  objective  of  the  competition  is  to  advance  robotic  technology  used  in  range  clearance 
operations  in  order  to  increase  operational  effectiveness  while  providing  greater  safety  for  range 
clearance  team  members.  There  are  four  events  that  will  be  judged  separately  and  then  an 
overall  prize  based  on  the  best  qualifying  overall  score. 

The  four  events  that  make  up  the  competition  are: 

1 .  Vegetation  removal 

2.  Surface  clearance 

3.  Geophysical  mapping 

4.  Sub-surface  clearance 


Organization 

The  judging  duties  are  distributed  across  six  official  judges  for  the  competition. 

These  are: 

Chief  Judge  -  Dr  Jim  Overholt  -  USA/TARDEC, 

Safety  Judge  -  Lucas  Martinez  -  USAF/AFRL, 

Autonomy  Judge  -  Mike  Bruch  -  USN/SPAWAR, 

Geophysical  mapping  Judge  -  Andy  Schwartz  -  USA/USACOE, 

Task  Judges  -  Walt  Waltz  USAF/AFRL  and  Plyer  McManus  USA/USACOE. 

The  duty  of  each  judge  is  to  oversee  the  events  for  your  responsible  areas  and  then  to 
Adjudicate  the  scoring  for  that  event.  You  will  have  data  collection  assistants  and  field  referees 
to  assist  you.  In  addition  we  will  be  collecting  video  of  as  much  as  possible  for  you  to  review  if 
needed. 

The  Chief  Judge  is  responsible  to  supervise  the  judges,  data  collection  assistants,  and  field 
referees,  verify  and  validate  the  scoring  of  all  events,  and  interpret  the  rules  as  applied  to 
competition  adjudication  matters. 

The  Safety  Judge  is  responsible  to  conduct  and  supervise  all  of  the  equipment  safety  tests, 
verify  and  validate  the  safety  test  methods,  and  observe  all  of  the  team  events  to  judge  any 
perceived  safety  violations  or  issues. 
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The  Autonomy  Judge  is  responsible  to  supervise  the  autonomy  data  collection  assistants  and 
observe  all  of  the  team  events  and  related  autonomy  data  to  judge  the  level  of  autonomy  of  the 
robotic  systems. 

The  Geophysical  mapping  Judge  is  responsible  to  supervise  the  geophysical  mapping  data 
collection  assistants,  verify  the  condition  of  the  event  site,  observe  all  of  the  geophysical 
mapping  events,  collect  the  geophysical  mapping  data  from  the  teams,  and  process/calculate 
the  geophysical  mapping  data  to  judge  the  performance  to  the  rules. 

The  Task  Judges  are  responsible  to  supervise  the  data  collection  assistants,  verify  the  condition 
of  the  event  sites,  observe  all  of  the  team  events,  inspect  the  condition  of  the  sites  and 
debris/collection  areas  post  event,  and  to  judge  the  task  performance  for  each  event. 


Training 

There  will  be  judge  training  for  the  1st  three  days  on  site  for  you  to  practice  and  get  your 
assistants  lined  up  prior  to  the  event.  We  will  have  surrogate  robotic  systems  and  operators  to 
perform  each  of  the  task  events.  The  expectation  is  to  conduct  trial  runs  and  have  the  judges 
do  mock  scoring  to  practice  and  work  out  any  details. 

Event  Activities 

For  the  actual  events,  the  judges  are  expected  to  be  on  site  at  least  one  hour  prior  to  the  start  of 
each  event.  The  events  will  last  up  to  nine  (9)  hours  from  the  start.  Due  to  the  nature  of  the 
event  and  the  available  assistants,  there  should  be  ample  time  for  the  judges  and  officials  to 
take  breaks  and  have  lunch.  Meals  and  water  will  be  provided  at  the  competition  site.  At  the 
completion  of  each  event  the  judges  will  be  responsible  to  collect  all  of  the  data  for  their 
event/area,  inspect  the  event  sites  and  collection  piles,  and  then  review  and  score  their  event. 
The  scores  will  be  submitted  to  the  chief  judge  who  will  review  the  data  and  then  release  the 
judges/officials  when  he  is  satisfied  with  the  score.  The  chief  judge  will  maintain  the  scores  and 
data  in  a  safe  location  for  record  keeping  post  event. 

Facilities 


The  judges  will  be  staying  at  the  Grey  Rocks  Ranch  facility.  There  is  a  conference  room  at  the 
ranch  that  can  be  used  for  deliberations  and  meetings  to  discuss  the  scoring  after  the  events. 
At  the  competition  site,  there  will  be  a  work  site  trailer  for  the  judges  to  use  for  their  data 
collection  and  on-site  needs. 


Please  refer  to  the  R2C2  Rules  and  metrics  that  you  have  been  provided  for  the  scoring 
breakdowns  and  official  rules,  and  the  R2C2  October  Validation  Trials  AAR  and  Recommended 
Rules  changes  draft  final  v.  DEC  2010 

Please  refer  any  questions  about  your  role  or  the  competition  event  to  Mr.  Brian  Skibba,  AFRL 
(850)819-6905. 
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SUBJECT:  Competitor  Letter  of  Intent  to  Participate  in  the  Robotic  Range  Clearance  Competition 

1.  Purpose.  OUSD  (AT&L)/PSA,  LW&M  is  planning  to  conduct  a  Robotic  Range  Clearance  Competition  and  to 
award  $2M  in  prizes.  This  Competitor  Letter  of  Intent  (LOI)  will  act  as  an  expression  of  intent  to  participate  in  the 
Robotic  Range  Clearance  Competition.  The  Competitor  team  or  organization  agrees  to  have  a  mid-progress  review 
and  appear  at  the  Competition  site  for  the  final  Competition  at  a  mutually  agreed  upon  time,  ready  to  perform.  This 
LOI  does  not  constitute  or  create,  and  shall  not  be  deemed  to  constitute  or  create,  any  legally  binding  or  enforceable 
obligation  on  the  part  of  either  party.  OUSD  (AT&L)/PSA,  LW &M  reserves  the  right  to  cancel  the  Competition  at 
any  time  (including  the  cancellation  of  any  cash  prizes)  if  dictated  by  Department  of  Defense  (DoD)  priorities, 
availability  of  funding,  lack  of  sufficient  participation  of  competitors,  or  for  any  other  reason  determined  to  be  in  the 
interests  of  DoD. 

2.  Background.  Based  on  the  positive  response  to  a  previous  Request  for  Information,  the  Office  of  Secretary  of 
Defense  (OSD),  in  collaboration  with  the  US  Army,  is  planning  to  conduct  a  Competition  (with  a  cash  prize  for 
the  winner(s))  to  foster  the  ability  to  clear  training  ranges  of  debris  and  unexploded  ordnance  using  robotic 
technologies.  DoD  intends  for  this  Competition  to  result  in  viable  systems  that  can  be  placed  into  service  clearing 
ranges.  To  better  plan  and  manage  logistical  requirements  for  this  Competition,  the  Competition  organizers 
request  that  companies  or  organizations  that  are  planning  on  entering  this  Competition  make  a  commitment  by 
signing  this  LOI.  The  Competition  organizers  will  publish  Competition  rules,  scoring  criteria,  and  any  rule 
changes  in  a  timely  manner. 

3.  Competitor  Understandings  and  Agreements-  The  points  of  understanding  and  agreement  are  listed  below: 

•  The  competitor  teams  or  organizations  agree  to  abide  by  the  rules  of  the  Competition  and  agree  to 
accept  the  judges’  rulings  as  final. 

•  The  Competition  will  be  held  at  Camp  Guernsey,  WY. 

•  Any  Government  Furnished  Equipment  (GFE)  made  available  to  competitors  will  be  on  a  first  come- 
first  serve  basis,  subject  to  availability  and  military  needs. 

•  If  the  competitor  becomes  unable  or  unwilling  to  take  part  in  the  Competition,  the  competitor  shall 
notify  the  competition  organizers  immediately  of  its  withdrawal.  If  the  competitor  withdraws  from  the 
Competition,  it  will  not  be  allowed  to  re-enter  the  Competition  and  its  position  in  the  Competition  will 
be  forfeited. 

4.  Team  Prime.  Each  team  planning  to  participate  in  the  Competition  shall  designate  a  Prime  member.  The 
Prime  will  be  the  point  of  contact  with  the  competition  organizers.  The  Prime  will  be  the  representative  who 
signs  this  LOI  for  the  team.  The  cash  awards  for  the  Competition  will  be  awarded  to  the  Prime.  In  the  event  that 
a  member  of  the  team  is  added  or  replaced,  the  Prime  is  responsible  for  obtaining  approval  via  the  competition 
organizers.  Organizations  can  participate  on  more  than  one  team,  but  they  may  ONLY  be  the  Prime  on  one  team. 

No  individuals  can  participate  on  more  than  one  team.  All  team  members  will  be  listed  on  Attachment  A. 

5.  Effective  date.  This  LOI  will  become  effective  when  signed  by  the  Prime  member  and  will  remain  in  effect 
until  the  final  date  of  the  Competition  (planned  for  calendar  year  2011).  This  LOI  must  be  submitted  to  the 
competition  organizers  no  later  than  5:00PM  (EDT)  03  May  2010. 


Team  Name 


Signature 


Date 


Organization 
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Team  N 

fame 

Company/Organization 

POC 

Email 

Phone 
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1.  Introduction 

The  Office  of  Secretary  of  Defense  (OSD)  in  collaboration  with  the  US  Air  Force  and  US  Army  is 
conducting  a  Robotic  Range  Clearance  Competition  (R2C2)  to  foster  the  ability  to  clear  training 
ranges  of  debris  and  unexploded  ordnance  (UXO)  using  robotic  technologies.  The  purpose  of 
R2C2  is  to  quickly  tap  into  the  innovation  and  ingenuity  of  the  commercial  robotic  technology 
sector  to  improve  the  safety  and  effectiveness  of  the  four  tasks  traditionally  associated  with  range 
clearing  operations:  1)  Vegetation  removal,  2)  Surface  clearance,  3)  Geophysical  mapping,  and  4) 
Subsurface  clearance. 

This  packet  is  intended  to  provide  to  competitor  teams  logistics  information  and  range  data  for  the 
competition. 
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2.  General  Information 

The  R2C2  events  will  be  conducted  on  the  northern  training  areas  at  Camp  Guernsey,  WY, 
approximately  25  miles  from  the  main  camp.  The  surrounding  area  is  comprised  of  mixed  grass  and 
sagebrush  upland  plains  with  sections  of  lodge  pole  pines  and  cottonwoods.  These  competition  sites 
are  all  contained  within  a  600-acre  continuous  area  at  an  elevation  of  approximately  5000  ft.  The 
sites  are  representative  of  military  impact  areas  and  training  ranges  that  are  in  need  of  unexploded 
ordnance  (UXO)  clearance.  Note:  this  area  has  no  known  history  of  UXO. 

2.1.  Schedule 

The  competition  will  be  conducted  from  August  7-15,  2011  with  a  Media/Visitor  day  scheduled  for 
Thursday  August  11.  R2C2  competitor  practice  and  setup  activities  will  be  conducted  August  1-5. 
Each  competition  event  will  be  conducted  over  an  8  hour  shift  to  represent  1  work  day  starting  at 
8am  until  5pm  local  time.  Depending  on  the  day,  up  to  two  teams  will  be  competing  simultaneously 
in  separate  events.  See  technical  data  section  for  exact  location  of  your  ranges. 


Table  1:  Major  Event  Schedule. 


Event 

Location 

Date 

Time 

Competitor  Check  In 

JTEC  Office 

8/1-8/7/2011 

Call  to  schedule 

Practice  Week 

Competition  Area 

8/1-8/5/2011 

0800-1700 

Kickoff/Opening  Reception 

Building  01 1 

8/7/2011 

1700 

Competition 

Competition  Area 

8/8-8/13/2011 

0800-1700 

Media  and  Visitor  Day 

Competition  Area 

8/11/2011 

0800-1430 
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2.2. 


Notional  Daily  Schedule 


Time 

Event 

0400 

Paddock  unlocked. 

0500 

Check-in  begins,  anyone  arriving  after  morning  briefing  checks  in  at 
Command  and  Control  before  coming  into  the  Paddock. 

0515 

Morning  safety  briefing  in  the  Mess  Tent. 

Equipment  can  be  moved  to  staging  areas  after  the  safety  briefing. 

0600 

Ranges  open  for  range  walks.  Breakfast  available  in  the  Mess  Tent. 

0800 

Competition  events  begin. 

1200 

Lunch  available  in  the  Mess  Tent. 

1700 

Ranges  closed. 

Move  all  equipment  back  to  Paddock 

1730 

Fueling  begins. 

1800 

Briefing  and  announcements  for  the  following  day  in  the 

Mess  Tent.  Dinner  available  in  the  Mess  Tent. 

2000 

Paddock  locked  for  the  night. 
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2.3.  Contact  Information 

Camp  Guernsey  Point  of  Contact:  Keith  Reedy:  (307)  331-3743 


Competition  Director: 

Brian  Skibba;  (850)  819-6905 

Competition  Logistics: 

Travis  Bruegger:  (937)  238-8288 

Competition  Operations: 

Dave  Malek:  (937)  684-6250 

Medical  Emergencies: 

911 

Fires/Other  Emergencies/Urgent  Matters  Relating  to  Camp  Guernsey  (24  hrs): 

Camp  Guernsey  Fire  Desk:  (307)  836-7810 
In  case  of  fire,  call  911  AND  the  Fire  Desk 
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2.4.  Requirements  for  Field  Operations 

Competitors  must  meet  the  following  requirements  before  they  will  be  allowed  to  operate  on  the 
competition  ranges. 

2.4.1.  Safety  Test 

All  vehicles  used  on  competition  ranges  must  pass  a  safety  test  before  they  will  be  allowed  to 
operate  any  vehicles  on  the  real  or  practice  ranges.  Safety  tests  can  be  completed  any  time  during 
practice  week.  See  the  SAFETY  GUIDANCE  &  REGULATIONS  document  for  details. 

2.4.2.  Environmental/Cultural  Briefing 

All  competitors  must  attend  an  environmental/cultural  briefing  before  conducting  any  operations  in 
the  competition  area.  This  is  required  for  all  individuals  participating  in  the  competition.  The 
briefing  will  be  held  at  the  Competition  Site  early  in  the  Practice  Week.  It  is  a  one-time  briefing 
and  should  last  no  more  than  one  hour.  When  on-site,  please  see  Command  and  Control  (C&C)  for 
more  information. 


2.4.3.  Confirm  Radio  De-confliction-  If  necessary,  during  the  competition 

If  any  radio  frequency  conflicts  are  identified  during  practice  week,  appropriate  measures  will  be 
implemented  to  mitigate  the  problem.  When  on-site,  please  contact 
Command  and  Control  for  more  information  or  to  report  a  conflict. 
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2.5.  Other  Advisories 

While  basic  necessities  will  be  provided,  the  competition  will  be  held  outside  so  please  be  prepared 
for  weather  conditions.  Range  elevations  are  around  5000  ft.  mean  sea  level  (MSL.)  The  effects  of 
ultraviolet  ray  exposure  from  the  sun  are  amplified  at  these  altitudes.  Typical  early  August  days 
are  hot  and  dry  (highs  in  the  90s)  with  cool  nights  (lows  in  the  60s)  and  possible  thunderstorms  in 
the  afternoons.  Please  dress  accordingly.  Wide  brimmed  hats,  sunscreen,  and  insect  repellant  are 
highly  recommended. 


All  participants  should  drink  plenty  of  water  to  prevent  dehydration.  The  R2C2  will  provide 
drinking  water  onsite. 
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3.  Housing  Logistics 


All  competing  teams  will  be  staying  in  barracks  in  the  Cantonment  area  at  CampGuemsey  during 


the  competition.  Teams  are  expected  to  arrive  between  August  1st  and  5th  during  nonnal  business 
hours.  Please  schedule  the  arrival  of  your  personnel  and  equipment  with  R2C2  Staff  prior  to  your 
arrival  so  someone  can  be  available  to  greet  you.  R2C2  staff  will  provide  you  with  room  keys  and 
a  briefing  when  you  arrive.  To  schedule  your  arrival,  please  contact  Paula  Frisbey  at  (307)  836- 
7839  or  pfrisbey@ara.com. 


Main  Entrance 
Parking  Lot 
Women's  Squad  Bay 
Dining  Facility 
C&S  Trading  Post 
Chapel 

Fitness  Center 
Men's  Squad  Bays 
JTEC  Office 


Figure  1:  Map  of  Cantonment  Area  at  Camp  Guernsey 


Access  to  the  camp  is  through  the  front  gate  only.  Guards  at  the  gate  will  check  ID’s  during  entry, 
but  no  visit  requests  are  required.  Drivers  of  vehicles  must  be  certified  to  operate  the  vehicles  they 
are  driving  (e.g.,  if  operating  a  vehicle  that  requires  a  commercial  driver’s  license  (CDL),  the 
operator  must  be  prepared  to  present  a  valid  CDL  when  entering  Camp  Guernsey). 
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The  Camp  Guernsey  front  gate  is  open  between  0500  and  2200  every  day.  For  after-  hours  access 
to  the  Cantonment  Area,  you  will  need  to  push  the  intercom  button  at  the  front  gate  to  request 
entry.  Large  vehicles  can  be  parked  temporarily  at  the  parking  lot  marked  on  the  map  shown  in 
Figure  1.  However,  all  competition  vehicles  and  trailers  should  be  moved  to  the  competition  area 
as  soon  as  possible  after  coordinating  that  movement  with  the  R2C2  team  (see  section  2.3  for 
Points  of  Contact.) 

All  competitor  teams  will  be  housed  in  the  camp  barracks  in  8  person  squad  bays.  Men  will  be  in 
building  501  and  women  in  building  408.  Cohabitation  is  not  allowed  on  Camp  Guernsey.  You 
will  have  access  to  your  rooms  at  any  time  after  initial  check  in.  Parking  is  available  adjacent  to 
the  buildings. 

Showers  and  coin  operated  laundry  machines  are  available  in  the  barracks.  Internet  access  should 
be  available  in  the  barracks  for  a  fee.  You’ll  find  more  information  in  the  squad  bay  or  by 
connecting  to  the  wireless  router. 

Please  obey  all  posted  speed  limits,  signs,  and  directions  while  on  Camp  Guernsey.  Please  note 
that  the  posted  speed  limit  in  the  Cantonment  area  is  20  mph  (10  mph  when  pedestrians  are 
present).  The  speed  limit  in  the  North  Training  Area  (site  of  the  competition)  is  30  mph. 


3.1,  Resources  available  in  Camp  Guernsey 


Dining  Facility  -  Meals  are  available  in  the  mess  hall  for  any  participants  who  are  not 

at  the  competition  area.  Anyone  using  the  dining  facility  will  sign  in  at  the  mess  hall  sign  in  table 

for  each  meal.  Details  on  this  procedure  will  be  provided  at  check  in. 

C&S  Trading  Post  -  Convenience  store.  Chanel  -  Check  schedule  for  religious  services.  Fitness 
Center  -  Open  24  hours. 

Barracks  -  Housing  for  all  competitors. 

Wi-Fi  -  Wireless  internet  access  is  often  available  in  the  barracks  for  a  fee.  Daily,  weekly,  and 
monthly  rates  are  available 
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3.2.  Resources  available  in  and  near  the  town  of  Guernsey 

3.2.1.  Hospital/Pharmacy 
Register  Cliff  Pharmacy; 

437  W  Whalen  Ave;  Guernsey,  WY  82214  (307)  836-9275 

Guernsey  Medical  Center: 

1  East  Whalen  St.,  Guernsey,  WY  (307)  836-2422 

Platte  County  Memorial  Hospital: 

201  14th  St.,  Wheatland,  WY  82201  (307)  322-3636 

3.2.2.  Gas  Stations 
Johnston's  Corner: 

550  W  Whalen  St.,  Guernsey,  WY  (307)  836-3155 

Shell: 

200  E  Whalen  St.,  Guernsey,  WY 

3.2.3.  Hardware 

Howshar  Appliance  &  Hardware: 

18  South  Wyoming  Avenue,  Guernsey,  WY  82214  (307)  836-2611 

3.2.4.  Food 

Nana ’s  Kitchen  -  Quality  Sit  Down  restaurant  with  —home  cooking!; 

37  North  Wyoming,  Guernsey,  WY  82214  (307)  836-2010 

Craz  v  Ton  v’s  -  Bar  and  Grill,  good  low  cost  grill  food,  smoky  bar  atmosphere;  Highway  26 
Guernsey,  WY  82214 
(307)  836-2317 

The  Riverview  Restaurant  -  Diner  Food 

800  West  Laramie,  Guernsey,  WY  82214  (307)  836-2300 

The  Lunch  Box  -  (Lunch  only) 

Just  outside  the  main  gate  of  camp  Guernsey 

B&F  Foods  -  (Grocery)  Open  0700  -  2000  Daily 
452  Whalen  St.,  Guernsey,  WY  82214  (307)  836-2266 

Johnston's  Corner  -Gas  Station  Restaurant:  Pizza,  Fired  Chicken,  Subs 
550  W  Whalen  St.,  Guernsey,  WY  82214  (307)  836-3155 
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*Please  note  that  the  speed  limit  in  the  town  of  Guernsey  is  30  mph.  The  speed  limit  drops  suddenly  when  entering 
town  on  Highway  26  and  the  speed  limits  are  strictly  enforced  in  Guernsey. 
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4.  Competition  Logistics 

The  competition  will  be  held  in  the  North  Training  Area  between  0800  and  1700  according  to  the 
schedule.  Directions  to  the  competition  area  from  the  Camp  Guernsey  Cantonment  area  (billeting 
area  for  teams)  are  available  in  the  included  document— Directions  from  Camp  Guernsey  Front 
Gate  to  Competition  Site.pdfll.  The  route  to  the  competition  area  and  back  to  Camp  Guernsey’s 
Cantonment  Area  will  be  clearly  marked  with  signs  for  your  convenience. 

The  Paddock  will  be  unlocked  at  0400  each  morning  and  participants  must  sign  in  at  the  Command 
and  Control  (C&C)  trailer  upon  arrival.  Anyone  leaving  the  competition  area  must  sign  out  at 
C&C  before  leaving  the  Paddock. 

There  will  be  a  morning  safety  briefing  every  competition  day  at  05 15.  The  briefing  will  review  all 
pertinent  safety  procedures  and  cover  all  activities  that  are  scheduled  for  that  day  and  any  changes 
to  the  schedule  or  procedures.  Any  team  participating  in  competition  activities  must  have  a 
representative  present  during  the  safety  briefing.  Everyone  in  the  Paddock  at  05 15  is  expected  to 
attend  the  briefing  and  anyone  entering  the  Paddock  after  the  briefing  is  expected  to  review  the 
briefing  notices  posted  at  the  C&C  trailer. 

All  access  to  competition  ranges  will  be  strictly  controlled  by  C&C.  Any  movement  of  personnel 
or  equipment  to  the  ranges  or  staging  areas  must  be  coordinated  with  R2C2 
C&C.  All  competitors  will  be  escorted  to  and  from  the  ranges  by  R2C2  staff. 

All  competitors  will  be  allotted  a  space  in  the  Paddock  as  shown  in  Figure  2. 

Competition  vehicles  and  all  other  equipment  should  be  stored  within  the  allotted  space.  Non¬ 
competition  vehicles  should  be  parked  in  the  parking  area  outside  of  the  fenced  Paddock. 

The  Paddock  is  fenced  and  will  be  monitored  by  security  throughout  the  competition  daily  from 
1900  at  night  to  0700  the  next  morning. 
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Figure  2:  Paddock  Area 
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4.1.  Resources  available  at  the  competition  field 

Fueling  Station — Fuel  will  be  provided  for  all  competition  vehicles. 

Restrooms —  A  restroom  trailer  will  be  available  in  the  Paddock  area. 

Electricity — 120VAC  is  available  to  competitors  inside  the  Paddock  from  generators  that  operate 
the  flood  lights  (labeled  -LT  SETH  on  the  map  in  Figure  2.) 

Tent — Each  team  will  be  provided  with  a  15’X15’  walled  tent  in  the  Paddock. 

Meals — Breakfast,  Lunch  and  Dinner  will  be  available  in  the  mess  tent  (See  Schedule,  Section 

2.1). 

Snacks  -  Snack  foods  and  bottled  water  will  be  available  at  all  times  in  the  mess  tent. 

Mobile  phone  coverage — Phones  operating  on  the  Verizon  Network  are  known  to  work  in  most  of 
the  competition  area,  including  3G.  Your  coverage  may  vary  depending  on  location. 

Internet — No  internet  connection  will  be  provided  to  teams  by  the  R2C2,  but  you  may  be  able  to 
access  data  services  through  your  cell  phone  service  provider  (at  your  cost). 
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4.2.  Field  Data 

Each  competitor  has  been  assigned  a  competition  range  for  each  event.  These  are  referred  to  as  the 
— Real!  ranges.  Competitors  shall  not  operate  any  equipment  on  the  Real  ranges  except  during  their 
scored  events.  Each  team  has  also  been  assigned  a  practice  range  for  each  event  that  they  will 
attempt.  Teams  are  free  to  operate  any  equipment  on  their  practice  ranges  in  preparation  for  the 
competition  events.  The  practice  ranges  approximate  the  conditions  on  the  real  ranges,  but  some 
variations  should  be  expected. 

Every  range  is  defined  by  four  corner  points  (Northwest,  Northeast,  Southeast,  and  Southwest).  All 
of  the  corner  points  and  additional  data  describe  below  will  be  provided  to  the  Teams  in  Shapefile 
file  format.  Shapefiles  will  include  2  sets  of  3  files  with  the  extensions:  .shp,  .shx,  and  .dbf.  The  set 
of  three  files  with  the  string  —151  in  the  file  name  define  the  boundary  lines,  while  the  files  with 
the  string  —  111  contain  the  points.  All  of  the  files  are  necessary  to  fully  define  the  competition 
range  data. 

GPS  coordinates  are  given  in  Universal  Transverse  Mercator  (UTM)  system,  zone  13N.  All  files 
will  be  named  after  their  respective  ranges  as  detailed  below. 

Character  1:  R:  Real 
P:  Practice 

Character  2:  #:  Specifies  range  number 

Character  3-5:  GEO:  Geophysical  Mapping  SUB:  Subsurface  Clearance  SUR:  Surface  Clearance 
VEG:  Vegetation  Removal 

Character  6-9:  _##:  numerical  identifier  added  automatically 
Example:  R2VEG15:  Real  Vegetation  Removal  range  2 

4.2.1.  Data  Nomenclature 

I  vocalization  Points:  Three  localization  points  will  be  provided  for  each  range  internal  to  the  SHP 
file  system.  Additionally  a  concrete  monument  has  been  planted  at  each  localization  point  to  allow 
competitors  to  calibrate  their  GPS  readings  to  the  values  measured  by  the  R2C2  team.  Localization 
points  are  named  with  a  three  character  identifier  and  one  digit  identifier  number. 


Character  1-3:  LOC:  Specifies  Localization  Point 
Character  4:  #:  Specifies  point  number 

Example:  LOC2  is  localization  point  2 
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Range  Corner  Points:  The  competition  ranges  are  defined  by  3  sets  of  4  points.  Range  Corner 
Points  define  the  scored  area  and  shall  be  named  after  their  two  character  cardinal  direction  that  is 
most  applicable  as  detailed  below.  Offset  Corner  Points 

indicate  the  comers  of  the  30  meter  offset  from  the  range  in  which  competitor  vehicles  are  allowed 
to  operate.  Extended  Offset  Comer  Points  indicate  the  33  meter  boundary  which  competitor 
vehicles  are  not  permitted  to  cross  (excluding  entry  sides  as  dictated  by  the  Rules  document). 

Character  1:  Z:  Specifies  Range  Corner  Points 
F:  Specifies  Offset  Comer  Point 
E:  Specifies  Extended  Offset  Corner  Point 

Character  2-3:  Two  character  after  the  most  applicable  cardinal  direction 
Example:  ZNW  is  the  northwest  comer  of  the  range 

Obstacle  and  Obstruction  Points:  These  points  consist  of  4  points  that  bound  an  obstruction 
which  vehicles  should  avoid.  These  points  are  named  with  an  alpha  identifier,  a  number  and  a  two- 
character  cardinal  direction  identifier.  Obstacles  are  stacks  of  wooden  railroad  ties  intentionally 
added  to  ranges.  Obstmctions  are  any  other  object  or  terrain  which  the  competitor  vehicles  should 
avoid. 

Character  1-3:  OBS:  Specifies  Obstacles 
OBT:  Specifies  and  Obstruction 
Character  4-5:  ##:  Specifies  point  number 

Character  6-7:  Two  characters  representing  the  most  applicable  cardinal  direction 
Example:  OBS2NE  is  the  northeast  corner  of  obstacle  2 

Dump  Area  Points:  These  points  consist  of  4  points  that  bound  the  dump  area  (used  for  the 
Surface  Clearance,  Sub  surface  Clearance,  and  Vegetation  Removal  Events.)  Dump  area  points  are 
named  with  an  alpha  identifier  and  a  two-character  cardinal  direction  identifier.  For  the  Subsurface 
and  Surface  Clearance  ranges,  Seed  Items  must  be  deposited  in  the  Dump  Area  for  scoring.  For  the 
Vegetation  Clearance  ranges,  the  dump  area  is  provided  as  a  suggested  drop  area  for  removed 
vegetation  if  needed.  See  the  Rules  and  Metrics  Document  for  additional  details. 

Character  1:  D:  Dump  Site 

Character  2-3:  Two  characters  representing  the  most  applicable  cardinal  direction 
Example:  DNE  is  northeast  comer  of  The  Dump  Site 
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4.3.  Provided  Competition  Data 

This  section  describes  the  data  for  each  range  type.  This  will  cover  infonnation  enclosed  in  the 
SHP  file  system  as  well  as  additional  types  of  files  included  in  each  competitor’s  custom  data 
packet. 


4.3.1.  Geophysical  Mapping 

The  Geophysical  mapping  range  is  a  relatively  flat  field  with  buried  metallic  seed  items. 
Competitors  are  expected  to  detect  and  locate  all  seed  items  in  the  range. 


4.3.1. 1.  Geophysical  Real  Range  Details 

•  The  Real  range  SHP  file  system  will  include  the  following  point  types: 

o  Localization  Points 
o  Range  Corner  Points 
o  Offset  comer  points 
o  Extended  Corner  Points 
o  Obstacle  Points 

•  The  Real  range  is  approximately  two  acres. 

•  The  Real  range  is  roughly  square. 

•  The  Real  range  contains  two  obstacles. 

4.3. 1.2.  Geophysical  Practice  Range  Details 

•  The  Practice  range  SHP  file  system  will  include  the  following  point  types: 

o  Localization  Points 
o  Range  Comer  Points 
o  Obstacle  Points 

•  The  Practice  ranges  are  roughly  one  acre. 

•  The  Practice  ranges  are  roughly  square. 

•  The  Practice  ranges  contain  one  obstacle. 

•  The  Practice  range  files  only  contain  corner  points,  localization  points  and  obstacle 
points. 

•  Competitors  will  be  provided  5  small  and  5  medium  Seed  Items  to  seed  their  practice 
ranges  as  necessary  to  validate  their  systems. 

•  No  seed  items  have  been  buried  in  the  practice  ranges.  Teams  should  bring  digging 
equipment  if  they  wish  to  bury  seed  items  on  their  practice  range. 
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4.3.2.  4  Subsurface  Clearance 

All  teams  competing  in  the  Subsurface  Clearance  event  receive  a  SHP  file  system  as  well  as  a  —Dig 
List!  pertaining  to  a  specific  assigned  Real  range.  This  —Dig  List!  is  a  Microsoft  Excel  2007  file  and 
contains  information  about  Seed  Item  size,  depth  and  horizontal  location.  Seed  Item  sizes  will  be 
listed  as  either  Medium  or  Large,  corresponding  to  the  Medium  and  Large  Industry  Standard 
Objects  (ISO)  being  used  in  this  event.  Seed  Item  Depths  (measured  to  the  center  of  mass  of  the 
Seed  Item)  will  be  given  in  the  following  ranges:  0-0.3  m,  0.2-0. 5  m,  0.4-0. 7  m,  and  0.6- 1.0  m.  No 
Seed  Items  will  be  buried  deeper  than 
1  meter. 


Location  is  given  in  Northing  and  Easting  accompanied  by  the  Horizontal  Error  margin  (for  both 
Northing  and  Easting),  using  the  UTM  zone  13N  state  plane  as  a  reference.  Collected  Seed  Items 
should  be  deposited  in  the  Dump  Area.  Refer  to  the  previously  distributed  — SEEDED 
METALLIC  ITEMS  DESCRIPTION  v.  8  FEB  201  111  document  for  descriptions  of  the  Seed  Items. 


4.3.2. 1.  Subsurface  Real  Range  Details 

•  The  Real  range  SHP  file  system  include  the  following  point  types: 

o  Localization  Points 
o  Range  Corner  Points 
o  Offset  comer  points 
o  Extended  Corner  Points 
o  Dump  area  Points 

•  The  Real  range  with  have  an  associated  —Dig  List!  Microsoft  Excel  2007  file. 

•  The  Real  range  is  approximately  two  acres. 

•  The  Real  range  is  roughly  square. 


4.3.2.2.  Subsurface  Practice  Range  Details 

•  The  Practice  range  SHP  file  system  include  the  following  point  types: 

o  Localization  Points 
o  Range  Comer  Points 
o  The  Practice  ranges  are  roughly  one  acre, 
o  The  Practice  ranges  are  roughly  square. 

o  Competitors  will  be  provided  5  Large  and  5  Medium  Seed  Items  to  seed  their 
Practice  ranges  as  necessary  to  validate  their  systems. 

•  No  seed  items  have  been  buried  in  the  Practice  ranges.  Teams  should  bring  digging 
equipment  if  they  wish  to  bury  seed  items  on  their  practice  range. 
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All  teams  competing  in  the  Surface  Clearance  event  received  a  SHP  file  system  only.  The  Real  and 
Practice  ranges  will  be  seeded  with  various  sizes  of  Seed  Items.  Collected  Seed  Items  should  be 
deposited  in  the  Dump  Area. 

4.3.3. 1.  Surface  Real  Range  Details 

•  The  Real  range  SHP  file  system  include  the  following  point  types: 

o  Localization  Points 
o  Range  Corner  Point 
o  Offset  comer  points 
o  Extended  Corner  Points 
o  Obstacle  Comer  Points 
o  Dump  area  Points 

•  The  Real  range  is  5  acres. 

•  The  Real  range  is  roughly  square. 

•  The  Real  range  contains  2  obstacles. 

•  The  Real  range  will  be  seeded  with  various  sizes  of  Seed  Items. 

4.3.3.2.  Surface  Practice  Range  Details 

•  The  Practice  range  SHP  file  system  include  the  following  point  types: 

o  Localization  Points 
o  Range  Corner  Points 

•  The  Practice  ranges  are  roughly  one  acre. 

•  The  Practice  ranges  are  roughly  square. 

•  The  Practice  range  will  be  seeded  with  10  each  of  small,  medium,  plate,  and  angle  Seed 
Items. 
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Robotic  Range 

Clearance  Competition  R2C2  Competitor  Information  Packet  v.  22  July  201 1 

4.3.4.  Vegetation  Clearance 

All  teams  competing  in  the  Vegetation  Clearance  event  received  a  SHP  file  system  only.  Each 
vegetation  clearance  range  has  areas  that  have  been  deemed  impassible  for 

large  equipment.  These  areas  have  been  labeled  as  obstructions  internal  to  the  SHP  file  system,  per 
the  nomenclature  section  (4.2.1).  Additionally  the  obstructed  areas  are  marked  with  high  visibility 
fence.  Competitors  should  not  cross  into  obstructions  and  competitors  are  not  responsible  for  any 
vegetation  removal  in  the  obstructed  areas.  A  large  dump  area  has  been  designated  outside  of  the 
vegetation  clearance  ranges.  The  dump  area  is  available  if  needed.  See  the  Rules  and  Metrics 
document  for  additional  information. 


4.3.4.1.  Vegetation  Real  Range  Details 

•  The  Real  range  SHP  file  system  will  include  the  following  point  types: 
o  Localization  Points 

o  Range  Comer  Points 
o  Offset  Corner  points 
o  Extended  Comer  Points 
o  Dump  area  Points 
o  Obstructions  Points 

•  The  Real  range  is  approximately  3  acres 

•  The  Real  range  is  not  square. 

•  The  eastern  half  of  the  Real  range  contains  all  of  the  trees  for  cutting  but  the  entire  area 
must  be  cleared  per  the  contest  rules. 

4.3.4.2.  Vegetation  Practice  Range  Details 

•  The  Practice  range  SHP  file  system  will  include  the  following  point  types: 
o  Localization  Points 

o  Range  Comer  Points 

•  The  Practice  ranges  are  roughly  one  acre. 

•  The  Practice  ranges  are  not  square. 
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H.l.  GEOPHYSICAL  MAPPING  CHALLENGE 


H.1.1.  Introduction 

The  geophysical  mapping  challenge  is  intended  to  demonstrate  the  following  geophysical 
mapping  objectives: 

•  Navigate  a  digital  geophysical  mapping  platform  within  the  designated  area  to  collect 
digital  geophysical  data  so  that  buried  metallic  objects  can  be  detected  and  located  to  30 
cm  or  better  positional  accuracy. 

•  Collect  raw  geophysical  data  with  an  objective  noise  level  to  be  determined  at  the  site. 

•  Deploy  a  time  domain  electromagnetic  induction  metal  detector  and  record  its  data  over 
100%  of  the  designated  area  at  a  line  spacing  of  50  cm. 

The  geophysical  mapping  range  was  constructed  over  an  area  of  6,455.6  square  meters.  Two 
obstacles  were  placed  within  the  area  reducing  the  required  coverage  area  to  approximately 
6,405  square  meters.  One  hundred  and  twenty-five  pipe  sections  were  buried  randomly 
throughout  area.  All  pipe  sections  were  buried  in  a  vertical  orientation,  which  produces  a  single 
mono-pole  anomaly  in  horizontal  loop  electromagnetic  induction  metal  detectors. 

Figure  H-l  shows  the  layout  of  the  geophysical  mapping  challenge  range  area. 

This  document  is  arranged  in  the  following  order:  Section  H-2  presents  the  geophysical  mapping 
metrics  provided  to  the  competitors  by  R2C2  in  the  Competition  Rules  and  Metrics  document 
and  modifications  to  the  geophysical  metrics  made  prior  to  the  start  of  the  competition  start. 
Section  H-3  presents  the  analysis  of  the  geophysical  data  performed  by  R2C2.  Section  FI-4 
presents  the  scoring  of  the  geophysical  data. 
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H.2.  GEOPHYSICAL  MAPPING  METRICS 


R2C2  Competition  Rules  and  Metrics  ;  Geophysical  Mapping  Metrics 

The  following  information  is  copied  from  the  R2C2  Competition  Rules  and  Metrics,  Version  10  , 
July  2011. 

H.2.1.  Delivery  of  Raw  data 

Raw  data  should  be  provided  to  the  judges  within  30  minutes  of  completing  the  geophysical 
mapping  event.  The  merged  data  should  be  delivered  to  the  R2C2  Oversight  Team  within  48 
hours  of  the  completion  of  the  geophysical  data  collection.  If  the  team  does  not  deliver  the 
merged  data  by  the  deadline,  the  score  for  the  entire  geophysical  mapping  event  shall  be  zero. 

H.2.2.  Noise  Level  (70  Points) 

Using  Oasis  Montaj’s  QC  tools  detennine  the  average  level  of  RMS  noise  in  the  raw  data. 
Baseline  noise  will  be  detennined  by  the  R2C2  Competition  Team. 

Score  will  be  determined  as  shown  in  Table  H-l 


Table  H-l.  Noise  Level  Scoring 


Noise  Range 

Points  Awarded 

Low  RMS  noise  level 

Most 

Moderate  RMS  noise  level 

Some 

High  RMS  noise  level 

Least 

H.2.3.  Sensor  Coverage  (60  points) 

Score  will  be  determined  by  the  percent  of  the  range  covered. 

Score  will  be  determined  by  the  following 
Percentage  of  coverage  multiplied  by  60  points  =  Score 
Less  than  95%  coverage  results  in  no  points. 

H.2.4.  Anomaly  Location  Accuracy  (60  points) 

Detennine  anomaly  peak  response  locations  using  Oasis  Montaj's  UX-Detect  automatic  anomaly 
detection  algorithm  on  the  submitted  merged  data. 

Number  detected  scoring 

(#  of  Detections  within  30cm  of  actual  burial  location  /  #  seeded  )*  60  =  detection  score 

H.2.5.  Survey  Speed  (60  points) 

Using  Oasis  Montaj,  the  point-to-point  velocities  will  be  calculated  for  all  data  collected. 

Scoring  will  be  based  on  percentage  of  the  data  collected  at  the  specified  design  speed. 

Design  speed  will  be  determined  by  competitor  team  prior  to  event. 

The  scoring  will  be  determined  by  the  following  table: 
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Table  H-2.  Survey  Speed  Scoring  Metrics 


Design  speed  range 

Points 

98%  to  100%  within  +  or  -  0.3MPH  of  design  speed 

60  points 

95%  to  98%  within  +  or  -  0.3MPH  of  design  speed: 

40  points 

75%  to  95%  within  +  or  -  0.3MPH  of  design  speed: 

20  points 

50%  to  75%  within  +  or  -  0.3MPH  of  design  speed: 

5  points 

less  than  50%  within  +  or  -  0.3MPH  of  design 
speed: 

0  points 

H.2.6.  Revisions  to  The  Geophysical  Metrics  Prior  to  Competition  Start 
H.2.6.1.  Noise  Level  (70  Points) 

No  changes 

H.2.6.2.  Sensor  Coverage  (60  points) 

The  scoring  was  revised  to  eliminate  a  zero  score  in  the  event  less  than  95%  of  the  area  was 
covered. 

All  teams  were  informed  the  method  of  calculating  coverage  score  would  be  following  the 
method  described  in  Attachment  A  to  this  document. 

All  teams  were  informed  they  would  not  be  penalized  for  missed  coverage  around  obstacles 
attributed  to  the  width  of  their  sensor  platfonns.  They  were  informed  polygons  would  be  drawn 
around  the  obstacles  to  account  for  their  platform  width  and  these  polygons  would  be  excluded 
from  their  coverage  scoring. 

The  area  to  be  mapped  was  reduced  by  20.5%.  Attachment  B  presents  the  infonnation 
communicated  and  provided  to  the  competitor  teams  prior  to  the  competition  start. 

H.2.6.3.  Anomaly  Location  Accuracy  (60  points) 

No  changes 

H.2.6.4.  Survey  Speed  (60  points) 

No  changes 
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H.3.  GEOPHYSICAL  DATA  ANALYSIS 


H.3.1.  Noise  Level 

The  purpose  of  the  noise  score  was  to  encourage  innovation  in  mitigating  noise  sources  from 
team’s  robots  as  well  as  the  deployment  form  factors  teams  selected  to  deploy  their  sensor 
packages.  Sensor  noise  due  to  geology  or  other  sources  not  attributed  to  the  robot  or  the 
deployment  fonn  factor  were  not  considered  in  this  scoring.  All  final  data  was  compared  visually 
and  via  simple  statistical  analysis  to  comfinn  competitor’s  processing  of  the  final  dataset(s)  did 
not  alter  the  noise  level. 

Noise  in  electromagnetic  induction  (EMI)  metal  detectors  used  to  detect  munitions  is  expected  to 
have  a  normal  distribution  about  a  mean  background  of  zero  response.  Noise  levels  were 
assessed  on  dynamic  geophysical  mapping  data  collected  in  the  yellow  polygon  shown  in  Figure 
H-l,  which  is  an  area  where  a  zero  mean  background  response  is  expected  because  no  metallic 
items  exist  in  this  polygon.  All  data  collected  by  each  competitor  within  this  polygon  was 
analyzed  for  the  following  statistical  parameters: 

•  Standard  deviation 

•  Range  of  values  between  the  five  percentile  and  the  ninety-five  percentile 

•  Goodness  of  fit  of  histogram  of  values  to  a  normal  distribution  of  background 
measurements 
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Figure  H-l.  Geophysical  Mapping  Challenge  Range  Area 
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The  first  two  statistics  were  measured  quantitatively.  The  last,  goodness  of  fit,  was  assessed 
qualitatively. 

To  arrive  at  a  score,  competitor’s  statistics  were  compared  to  baseline  statistics  calculated  from 
data  collected  by  R2C2  using  a  man-portable  EM61-MK2  in  the  background  area.  The  R2C2 
data  collection  represented  current  state  of  the  art  in  EMI  metal  detecting  for  munitions.  Scores 
were  generated  based  on  the  percent  difference  of  the  two  quantitative  statistics  compared  to  the 
R2C2-collected  EM61-MK2  baseline  statistics.  Assessing  the  goodness  of  fit  of  values  to  a 
normal  distribution  of  background  measurements  was  an  informal  approach  to  assess  normality, 
and  was  used  to  corroborate  the  quantitative  statistics  results. 

All  three  competitor’s  performance  in  minimizing  noise  exceeded  the  R2C2  baseline 
performance.  Table  H-l  presents  the  noise  statistics  results. 

Team  D4C  and  Team  Sky  Research  each  used  one  or  more  EM61-MK2  sensors  manufactured  by 
Geonics,  Ltd.  Team  UXOD  used  a  Dynamic  Nano  TEM  (DNT)  sensor  manufactured  by  Zonge 
International.  The  EM61-MK2  and  DNT  do  not  have  the  same  operating  characteristics  and  do 
not  measure  the  same  time  windows  in  the  time-domain  EMI  response.  However,  channel  one  of 
the  EM61-MK2  is  centered  at  approximately  266  microseconds  after  transmitter  turn-off  and 
channel  30  of  the  DNT  is  centered  at  approximately  229  microseconds  after  turn-off.  As  this 
competition  is  about  robotics  and  the  purpose  of  the  noise  score  is  to  assess  mitigation  of  robot 
and  deployment  fonn-factor  noise  sources,  the  methods  of  assessing  each  team’s  noise 
characteristics  described  above  and  summarized  in  Table  H-3  are  appropriate  for  these  purposes. 
They  address  the  question:  “Does  the  robot  or  its  sensor  deployment  fonn-factor  introduce  noise 
in  the  geophysical  instrument  measurements?”  Note:  these  statistics  are  not  appropriate  for 
assessing  geophysical  detection  perfonnance.  For  illustrative  purposes  only,  Table  H-3  also 
shows  data  from  Team  Sky  Research’s  sensor  #3  which  shows  noise  attributed  to  the  deployment 
fonn  factor.  Figure  H-2-Figure  H-6  show  the  histogram  distribution  of  background 
measurements  for  the  data  tabulated  in  Table  H-3. 

Team  Sky  Research  submitted  five  sensor  datasets.  The  center  sensor  dataset  was  assessed  for 
noise  characteristics  after  it  had  been  used  for  other  scoring  purposes.  Since  its  noise 
characteristics  were  poor,  all  other  sensor  datasets  were  assessed  for  noise  perfonnance.  Sensor  1 
had  the  best  noise  characteristics.  Sensor  1  and  Sensor  3  noise  characteristics  are  presented  in 
Table  H-l. 
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H.3.2.  Sensor  Coverage 

Sensor  coverage  was  assessed  using  the  Geosoft  “UCEFOOTPRINTCOV”  processing  routine. 
This  routine  calculates  percent  coverage  using  a  user-defined  circular  sensor  footprint.  This 
routine  does  not  automatically  compensate  for  along-track  sampling  gaps.  Therefore  the  average 
along-line  sampling  gaps  were  calculated  for  each  design  speed  used  by  each  team.  Each  average 
along-line  spacing  was  then  used  to  calculate  the  circular  footprint  needed  to  account  for  proper 
line  spacing  assessments.  The  line  spacing  requirement  was  0.5  m.  Figure  H-7  illustrates  this 
scenario  and  presents  the  information  used  to  select  appropriate  footprint  values  for  each  design 
speed  used  by  competitors.  Table  H-4  presents  the  design  speeds  used  by  each  competitor,  the 
footprint  diameter  used  to  assess  meeting  the  0.5  m  coverage,  and  the  coverage  percentage 
calculated  using  the  Geosoft  routine.  Figure  H-8-Figure  H-10  show  the  Geosoft  generated  maps 
illustrating  the  coverages  achieved. 


a=.25m 


r=Va2+b2  (  j  =0-5m  diameter  circle  centered 

on  hypothetical  measurement 
locations 


Line  spacing=0.5m 


Table  of  footprint  values  to  use  for 
various  along-line  sampling  intervals 

b=  along-line 
sampling 
interval 


.25 

.2 

0.320 

0.640 

.25 

.11 

0.270 

0.546 

.25 

.1 

0.269 

0.539 

.25 

.09 

0.266 

0.531 

.25 

.08 

0.262 

0.525 

.25 

.06 

0.257 

0.514 

Figure  H-7.  Input  Calculation  for  Geosoft  “UCEFOOTPRINTCOV” 
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Table  H-4.  Results  of  Coverage  Assessments 


Team 

Design  Speed 

Average  along- 
line  sample 
spacing 

Adjusted 

Footprint 

Diameter  for 

Coverage 

Calculation 

Calculated 

Coverage 

Team  D4C 

1  m/sec 

0.09  m 

0.531  m 

3,740.59  nr  - 
58.4% 

1.2  m/sec 

0.09  m 

0.531  m 

Team  Sky 
Research 

1.3  m/sec 

0.11  m 

0.546  m 

Sensor  #1; 

2,565.6  m2  = 
40.1% 

1.25  m/sec 

0.09  m 

0.531  m 

0.97  m/sec 

0.08  m 

0.525  m 

Team  UXOD 

1.34  m/sec 

0.09  m 

0.531  m 

83.9% 

516280  516300  516320  516340  516360  516380  516400 

Figure  H-8.  Coverage  Calculation  Map  for  Team  D4C 
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Figure  H-9.  Coverage  Calculation  Map  for  Team  Sky  Research 
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Figure  H-10.  Coverage  Calculation  Map  for  Team  UXOD 


193 

Distribution  A:  Approved  for  public  release;  distribution  unlimited.  88ABW-2012-5009,  19  September  2012 


H.3.3.  Anomaly  Location  Accuracy 

One  hundred  twenty-five  pipe  sections  were  buried  throughout  the  geophysical  mapping  area. 

All  were  placed  in  a  vertical  orientation  so  that  simple,  single-pole  anomalies  would  be  produced 
in  horizontal  look  EMI  sensor  data.  All  seeded  pieces  of  pipe  were  placed  at  depths  that  produce 
large,  clearly  detected  anomalies  in  all  digital  geophysical  mapping  systems  used  in  the 
munitions  response  industry.  Scoring  was  perfonned  only  against  the  125  seeded  pipe  sections. 
Any  existing  metallic  items  present  within  the  geophysical  mapping  area  were  not  included  in 
the  scoring. 

Anomaly  location  accuracy  was  calculated  using  the  UCEPROVE  Geosoft  routine.  This  routine 
compares  ground  truth  locations  to  detected  anomaly  locations  and  assesses  how  far  the  two  are 
from  each  other.  Anomalies  were  selected  from  processed  final  data  delivered  by  each 
competitor.  No  adjustments  were  made  to  any  measurement  locations  delivered  by  the 
competitors.  Table  H-5  summarizes  the  number  of  anomalies  detected  by  each  team  and  the 
number  found  within  the  30cm  requirement.  All  anomalies  detected  by  Team  UXOD  (125  total) 
and  Team  Sky  Research  (58  total)  were  detected  within  the  30  cm  requirement.  Team  D4C 
detected  97  anomalies  of  which  92  were  detected  within  the  30  cm  requirement.  The  five 
detected  but  not  within  the  30cm  requirement  were  beyond  the  30  cm  requirement.  A  qualitative 
analysis  of  Team  D4C  data  suggests  the  five  anomalies  detected  outside  the  30  cm  requirement 
are  due  to  gaps  in  coverage— portions  of  the  mapping  work  where  survey  lines  did  not  meet  the 
50  cm  line  coverage  metric.  Figure  H-l  1-Figure  H-13  show  the  Geosoft  generated  maps 
illustrating  the  anomaly  location  accuracy  analysis  results. 


Table  H-5.  Results  of  Anomaly  Location  Accuracy  Assessments 


Team; 

Team  D4C 

Team  Sky 

Research 

Team  UXOD 

Total  number  of  seed  items 
detected 

97 

69 

125 

Number  of  seed  items 
detected  within  30cm 

92 

69 

125 

Percent  of  all  possible  seed 
items  detected  within  30cm 

73.6% 

55.2% 

100% 
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Figure  H-12.  Anomaly  Location  Accuracy  analysis  for  Team  Sky  Research 
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Figure  H-13.  Anomaly  Location  Accuracy  Analysis  for  Team  D4C 
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H.3.4.  Survey  Speed 


Survey  speed  was  calculated  using  the  UCEVELOCITY  routine  in  Geosoft.  This  routine  calculates  the  point-to- 
point  distances  between  adjacent  measurement  locations  and  then  calculates  the  velocity  between  those  points 
based  on  the  time  taken  to  traverse  that  distance.  No  adjustments  were  made  to  any  measurement  locations 
delivered  by  the  competitors.  Table  H-6  summarizes  the  percent  of  the  mapping  perfonned  within  the  team- 
selected  design  survey  speeds.  Team  D4C  selected  two  design  speeds  during  their  mapping  event.  Team  Sky 
Research  selected  three  design  speeds  and  Team  UXOD  selected  one.  Figure  FI- 14  and  Figure  H-15  show  the 
survey  speed  analysis  for  Team  D4C.  Figure  H- 16-Figure  H-18  show  the  survey  speed  analysis  for  Team  Sky 
Research.  Figure  H-19  shows  the  survey  speed  analysis  for  Team  UXOD. 


Table  H-6.  Results  of  Survey  Speed  Assessments 


Team; 

Team  D4C 

Team  Sky  Research 

Team  UXOD 

Survey  Design  Speed  and 
Percent  Mapping 

Performed  at  Design  Speed 
±0.1341  m/sec  (±0.3MPH) 

1  m/sec;  91.34% 

1.2  m/sec;  75.59% 

1.3  m/sec;  80.93% 

1.25  m/sec;  89.04% 

0.97  m/sec;  87.99% 

1.34  m/sec;  58.81% 

Composite  Percentage 

Greater  than  75.59%  and 
less  than  91.34% 

Greater  than  80.93%  and 
less  than  89.04% 

58.81% 
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H.4.  SCORING 


The  scoring  was  performed  as  described  in  Appendix  C. 

The  geophysical  mapping  objectives  included  the  intent  to  deploy  a  time  domain  electromagnetic 
induction  metal  detector  and  record  its  data  over  100%  of  the  designated  area  at  a  line  spacing  of 
50  cm.  All  competitors  brought  geophysical  mapping  solutions  that  could  deploy  more  than  one 
sensor  simultaneously.  On  4  August  the  R2C2  verified  with  each  team  the  intent  was  score  the 
dataset  from  a  single  sensor.  On  August  4,  while  delivering  the  Q&A  document  (Attachment  A), 
and  again  on  August  6,  during  an  all-teams  briefing  called  by  the  R2C2,  all  teams  were  informed 
they  could  deploy  more  than  one  sensor  if  they  desired  and  either  themselves  select  which 
sensor’s  dataset  they  would  submit  for  scoring,  or  they  could  submit  all  sensor  datasets  and  the 
R2C2  would  score  each  individually  and  use  the  dataset  that  produced  the  highest  score  for  the 
geophysical  mapping  challenge  task  perfonnance  score.  Team  D4C  and  Team  UXOD  each 
submitted  a  single  dataset  for  scoring.  Team  Sky  Research  submitted  two  final  datasets,  one  for 
the  center  sensor  of  their  five-sensor  system,  and  one  containing  all  five  sensor’s  data.  Team  Sky 
Research  did  not  specify  which  dataset  they  wanted  scored.  The  R2C2  decided  to  process  each 
sensor’s  data  and  used  the  sensor  that  produced  the  highest  score.  Sensor  #1  scored  the  highest 
after  summing  the  individual  task  performance  scores  in  the  four  scoring  categories,  and  is  the 
sensor  used  for  the  Team  Sky  Research  score  reporting  below. 

Table  H-7  presents  the  geophysical  task  performance  scoring  for  each  team. 


Table  H-7.  Geophysical  Mapping  Task  Performance  Scores 


Coverage 


Anomaly 

Detections 


Speed 


Noise 

(units  are  mV) 


R2C2 

Baseline 

D4C 

%diff 

Sky  Sensor 

1 

%diff 

UXOD  Ch 
30 

%diff 

StdDev 

0.84 

0.84 

2.2 

-0.37 

2.57 

ave: 

0.00% 

-4.81% 

-2.41% 

0.82 

2 

-0.55 

2.55 

ave: 

-2.38% 

-5.56% 

-3.97% 

0.31 

0.5 

-0.96 

1.46 

-63.10% 

-45.93% 

-54.51% 

95%ile 

1.4 

5%ile 

-1.3 

Ranae 

2.7 

Score  (max.  70) 

70 

70 

70 

Percent 

Coveraae 

58.400% 

40.100% 

83.960% 

Round-up  plus  5 

64% 

46% 

89% 

percent  of  60 
points 

38.4 

27.6 

53.4 

Score  (max.  60) 

39 

28 

54 

Total  Detected 

92 

69 

125 

Percent 

Dete  cted 

73.6% 

55.2% 

100.0% 

Score  as 
percent  of  60 
points  based  on 
percent 
detected  within 
reauirement 

44.16 

33.12 

60 

Percent  within 
tolerance 

84.7% 

84.0% 

58.8% 

Score  (max.  60) 

20 

20 

5 

Total 

173.16 

151.12 

189 

205 

Distribution  A:  Approved  for  public  release;  distribution  unlimited.  88ABW-2012-5009,  19  September  2012. 


H.4.1.  Noise  Level 


The  purpose  of  the  noise  level  score  was  to  assess  noise  introduced  to  the  geophysical  data  due 
to  the  robot  or  the  deployment  fonn-factor  designed  into  the  robotic  geophysical  mapping 
solution.  Noise  attributed  to  other  sources  such  as  geology  or  geophysical  system  electronics 
were  not  a  factor  in  this  scoring.  Note:  no  sources  of  noise  were  observed  that  were  attributed  to 
geology  or  geophysical  system  electronics.  Noise  level  scoring  used  a  process  that  calculated  the 
percent  difference  between  various  statistics  describing  team  collected  data  and  statistics 
describing  data  collected  by  R2C2.  The  average  percent  difference  was  then  subtracted  from  100 
and  the  difference  multiplied  by  70  (the  maximum  points  allotted  to  noise  scoring.)  Negative 
percent  differences  indicate  the  team’s  sensor  noise  was  less  than  the  R2C2  baseline  noise.  All 
teams  had  negative  percent  differences  illustrating  all  used  deployment  platforms  on  their  robots 
that  minimized  noise  attributed  to  the  robot  or  the  deployment  platform.  Accordingly  all  three 
teams  were  allotted  the  full  70  points  for  their  noise  score. 

H.4.2.  Sensor  Coverage 

Sensor  coverage  was  calculated  as  the  percent  of  the  area  covered  multiplied  60,  the  maximum 
points  allotted  to  senor  coverage  scoring.  The  raw  percent  covered  calculated  was  adjusted 
upward  by  5%  for  each  team.  This  was  done  to  because  of  small  random  variations  typical  of 
GPS  systems  deployed  in  dynamic  modes  over  rough  terrain.  These  small  random  GSP 
variations  do  not  accurately  reflect  the  coverage  achieved  by  the  geophysical  sensors.  The  five 
percent  adjustment  provides  for  coverage  assessments  that  more  accurately  reflect  the  true 
coverage  achieved  by  the  geophysical  sensor. 

H.4.3.  Anomaly  Location  Accuracy 

Anomaly  location  accuracy  was  scored  as  the  percent  of  anomalies  detected  within  the  30  cm 
requirement  multiplied  by  60,  the  maximum  points  allotted  to  anomaly  location  accuracy 
scoring. 

H.4.4.  Survey  Speed 

Survey  speed  was  scored  according  to  the  table  presented  in  Section  H.3.4  above.  The  average 
speeds  tabulated  in  Table  H-6  were  used  to  assess  the  survey  speed  score. 
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H.5.  SUMMARY 


This  report  presents  the  metrics  and  scoring  of  the  task  perfonnance  portion  of  the  Robotic 
Range  Clearance  Competition’s  geophysical  mapping  challenge.  Geophysical  data  from  the  three 
competitors  were  analyzed  following  the  methods  outlined  in  the  final  R2C2  Rules  and  Metrics 
document  (Appendix  C).  All  data  analysis  was  performed  in  the  same  manner  for  each  of  the 
three  competitor’s  data.  All  scores  were  determined  using  the  processes  described  in  Appendix 
C.  Table  H-7  presents  the  geophysical  mapping  task  performance  scores  for  each  of  the  three 
competitors. 
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H.6.  ATTACHMENT  A 


Copy  of  Coverage  Calculation  and  Scoring  Method  Provided  to  Competitor  Teams  Prior  to 
Competition  Start. 


August  6,  201 1 

Robotic  Range  Clearance  Competition 

Geophysical  Mapping  Challenge 

Q:  Regarding  coverage  calculation  for  a  0.5m  design  specification,  how  are  small  deviations 
from  the  design  handled  in  the  scoring? 

A:  The  coverage  scoring  is  modified  in  response  to  this  question.  Score  will  be  detennined  by 
the  following: 

A.  Score  will  be  determined  by  the  percent  of  the  range  covered  at  the  50cm  design  line 
spacing.  All  coverage  calculations  will  exclude  areas  under  obstacles. 

B.  Score  will  be  determined  using  the  following  formula: 

a.  The  Geosoft  “UCEFOOTPRINTCOV”  GX  will  be  used  to  calculate  percent 
coverage  using  a  0.5m  footprint  input  value.  The  percent  coverage  will  be 
rounded  up  to  the  next  integer  value. 

b.  Integer  values  greater  than  94%  will  be  considered  to  fully  meet  the  coverage 
requirement  and  the  final  coverage  score  will  be  60. 

c.  For  integer  values  equal  to  or  less  than  94%,  the  integer  value  will  be  adjusted 
upward  by  5  percentage  points. 

d.  The  adjusted  percent  value  from  (c)  will  be  multiplied  by  60  and  the  product 
rounded  up  to  the  next  integer  value. 

e.  The  integer  value  from  (d)  will  be  the  final  coverage  score. 

C.  Examples: 

a.  A  team  achieves  a  coverage  of  94.2%  using  the  Geosoft 
“UCEFOOTPRINTCOV”  GX  and  a  0.5m  footprint  input  value  for  the 
calculation.  Ninety-four  point  two  is  rounded  up  to  95%,  which  is  greater  than 
94%.  The  final  coverage  score  is  therefore  60  points. 

b.  A  team  achieves  a  coverage  of  78.2%  using  the  Geosoft 
“UCEFOOTPRINTCOV”  GX  and  a  0.5m  footprint  input  value  for  the 
calculation.  Seventy-eight  point  two  is  rounded  up  to  79%.  The  final  coverage 
score  is  calculated  as:  (79%  +  5%)  x  60  =  84%  x  60  =  50.4,  which  is  rounded  up 
to  5 1 .  The  final  coverage  score  is  therefore  5 1 . 
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H.7.  ATTACHMENT  B 


Copy  of  Revised  Geophysical  Mapping  Area 


August  6,  201 1 

Robotic  Range  Clearance  Competition 

Geophysical  Mapping  Challenge 

To  better  meet  the  R2C2’s  objectives  of  having  all  competitors  complete  a  designated  area 
within  the  allotted  time,  the  northern  boundary  of  the  geophysical  mapping  area  has  been  revised 
to  reduce  the  total  area  over  which  competitors  will  be  scored.  The  new  northeast  and  northwest 
mapping  area  coordinates  are: 

Revised  Northeast  Corner:  5 16,379.0  East  4,699,292.0  North 

Revised  Northwest  Corner:  5 16,287.2  East  4,699,290.0  North 

All  coordinates  are  UTM  Zone  13  North,  units  are  in  meters. 

Competitors  are  not  required  to  map  the  northern  portion  that  has  been  deleted.  Any  mapping 
performed  in  the  deleted  area  will  not  be  used  in  any  manner  for  scoring  or  other  purposes.  The 
original  Area  of  Operation  boundary  and  the  original  Buffer  boundary  remain  unchanged. 
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Document  Change  Summary 


Section 

Description 

Date 

2.3.3 

Answered  all  previously  unanswered 
questions  and  moved  to  2.2.2. 

11  March  2010 

2.2.3.43 

Added  question  and  answer  re:  AUVSI 
schedule. 

18  March  2011 

2.2.1.10 

Added  question  and  answer  re:  potential 
government  shutdown. 

18  March  2011 

2.2.2.28 

Added  question  and  answer  re:  density  of 
seeded  items. 

18  March  2011 

2.2.2.29 

Added  question  and  answer  re:  magnetic 
sifters. 

18  March  2011 

2.2.2.30-39 

Added  questions  regarding  competition 
rules. 

19  MAY  2011 

2.2.2.40-44 

Added  questions  related  to  safety  testing 
rules. 

27  July  2011 

2.2.3.44-50 

Added  questions  related  to  competition 
procedures. 

27  July  2011 

2.2.2.45 

Added  a  question  related  to  Army 
geophysical  mapping  QC  requirements 

6  AUG  2011 
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1.  EXECUTIVE  SUMMARY 

1.1.  Scope 

This  document  lists  questions  posed  by  kickoff  meeting  attendees,  Industry  Day  meeting 
attendees,  and  other  potential  competitors  in  the  Robotic  Range  Clearance  Competition 
(R2C2)  about  the  competition  rules,  logistics,  scoring,  etc.,  and  their  answers. 

1.2.  Purpose 

The  purpose  of  this  document  is  to  establish  a  reference  for  questions  that  competitor  teams 
have  asked  and  the  answers  provided  by  the  R2C2.  The  intent  is  to  periodically  update  this 
document  as  questions  are  added  or  as  answers  change. 


It  is  important  to  note  that  these  answers  are  intended  to  be  informational  and 
supplement  other  sources  of  information  about  the  R2C2  competition.  The  R2C2 
Competition  Rules  and  Metrics  document  acts  as  the  authoritative  source  for 
competition  rules  and  metrics.  In  the  event  of  a  conflict  between  the  text  of  this 
document  and  the  current  R2C2  Competition  Rules  and  Metrics  document,  the  text  of 
the  current  version  of  the  R2C2  Competition  Rules  and  Metrics  document  takes 
precedence. 
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2.  COMPETITION  QUESTIONS  AND  ANSWERS 

2.1.  Questions  related  to  the  IDIQ 

2. 1 . 1 . 1  Will  the  IDIQ  be  a  service  contract  or  equipment  supply? 

ANS:  Acquisition  decisions  will  be  made  after  market  research.  Current  expectations 

are  an  IDIQ  service  contract. 

2. 1 . 1 .2  What  are  the  IDIQ  evaluation  criteria? 

ANS:  TBD.  Typical  criteria  include  cost,  quality,  schedule,  experience,  capability,  past 

performance.  The  Performance  Work  Statement  (PWS)  =  End  state,  not  how  to. 

2. 1 . 1 .3  What  is  the  size  of  the  IDIQ? 

ANS:  TBD.  Maybe  $40M. 

2. 1 . 1 .4  How  does  the  competition  criterion  differ  from  the  IDIQ  contract  criteria, 
particularly  as  it  relates  to  the  level  of  autonomy? 

ANS:  The  prize  competition  is  designed  to  push  autonomous  solutions  that  are  safe, 

cost  effective  and  deliver  the  necessary  level  of  quality.  The  IDIQ  contract  is 
seeking  similar  outcomes  therefore  the  criteria  will  be  similar. 

2. 1 . 1 .5  What  contract  vehicles  are  expected  to  provide  the  bulk  of  the  work  in  the  next 
5  years?  Is  WERS  expected  to  be  a  big  player?  Or  will  the  IDIQ  support  the 
UXO  clearance  market? 

ANS:  The  Government  will  use  all  available  vehicles  as  to  its  best  interest. 

2. 1 . 1 .6  Can  you  layout  the  difference  in  Service  market  size?  Has  the  Army  already 
spoken  to  present  vendors  on  this  robotics  need? 

ANS:  No  to  the  first  question,  and  yes  to  the  second. 

2. 1 . 1 .7  Will  the  USAEC  explain  the  relationship  of  the  Competition  and  the  selection 
criteria  for  the  IDIQ? 

ANS:  IDIQ  will  evaluate  experience,  capability  and  cost.  We  likely  won’t  separately 

evaluate  autonomy.  The  IDIQ  contract  will  require  unmanned  operation.  Teleop 
is  required;  autonomy  will  add  productivity. 
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2. 1 . 1 .8  Will  you  provide  cost  data  for  range  clearance  services  that  are  competitive 
today?  This  is  requested  so  we  will  know  if  our  automation  is  cost  effective  for 
the  ID  IQ. 

ANS:  Yes.  We  can  provide  money  and  time  to  perform  the  various  range  clearance 

tasks. 

2. 1 . 1 .9  Is  there  any  possibility  that  the  IDIQ  RFP  would  be  restricted  to  small  business? 

ANS:  Yes.  See  FAR  available  on  the  internet  for  small  business  requirements. 

2.1.1.10  Will  the  IDIQ  contract  be  a  MATOC  or  SATOC? 

ANS:  Market  research  will  dictate. 

2.1.1.11  Do  you  have  to  be  involved  in  the  prize  competition  to  bid  on  the  IDIQ? 

ANS:  No.  The  prize  competition  will  provide  an  avenue  for  demonstrating  past 

performance. 

2.1.1.12  Are  you  going  to  be  looking  for  night  operations  on  a  UXO  site? 

ANS:  Yes.  Range  schedule,  economy.  Range  schedules  often  limit  day  light  access  yet 

provide  free  access  at  night. 
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2.2.  Questions  Related  to  the  Competition 

2.2.1.  Competition  Goals 

2.2. 1 . 1  Any  thought  on  funding  upfront  costs  and  lowering  the  prize? 

ANS:  Funding  cannot  be  provided  up  front  to  lower  the  costs  to  competitors.  Unlike 

the  DARPA  challenges,  which  did  provide  funding  to  selected  competitors,  a 
contract  may  be  awarded  to  one  of  the  competitors  in  this  prize  competition. 
Though  the  IDIQ  award  is  a  separate  action  from  the  competition,  the 
Government  is  not  allowed  to  fund  an  effort  that  may  ultimately  be  submitted  in 
response  to  a  procurement  action.  To  help  alleviate  the  cost  burden  to 
competitors  the  Qualification  Trials  to  be  held  at  Camp  Guernsey  have  been 
cancelled  and  replaced  by  an  In-Process  Review  at  the  competitor  team  site. 

2.2. 1 .2  Can  you  publish  the  reports  documenting  the  work,  production,  lessons  learned, 
what  worked  and  what  doesn’t ,  strategies  used,  etc.  ? 

ANS:  There  is  be  a  public  release  approved  document  available  online  at 

www.roboticrangeclearance.com  that  captures  prior  work  AFRL  has  done  in  this 
domain. 

2.2. 1.3  If  existing  tele-op  systems  are  already  more  cost-effective  than  manual  system, 
why  require  autonomy  in  t  he  competition?  Shouldn’t  tele  -op  be  allowed  to 
compete? 

ANS:  Tele-operated  systems  are  invited  to  participate  in  the  competition.  However,  the 

competition  is  to  advance  technology  and  find  a  balance  between  the  tele-op 
systems  that  are  now  available  and  more  autonomous  approaches  that  increase 
cost  effectiveness,  productivity,  and  safety.  We  believe  the  R2C2  Rules  and 
Metrics  document  reflects  this. 

2.2. 1.4  It  appears  to  me  that  an  autonomous  robot  would  still  require  the  same  amount 
of- handlers!  as  a  tele-op  capability,  i.e.,  more  expensive  but  no  savings 
in  manpower.  So,  why  the  push  for  autonomous? 

ANS:  The  Government  believes  there  are  significant  savings  in  automated  robotics 

operations. 

2.2. 1 .5  The  competition  should  be  designed  either  for  a  FUDS  type  clearance,  a  range 
clearance,  or  a  range  construction  mission. 

ANS:  The  competition  will  be  designed  to  simulate  a  range  clearance  operation. 

2.2. 1.6  Is  there  a  preference  to  a  single  vehicle  vs.  multiple  vehicles  to  complete  the 

tasks? 
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ANS:  No. 

2.2. 1.7 

ANS: 

2.2. 1.8 
ANS: 

2.2. 1.9 

ANS: 

2.2.1.10 

ANS: 


AFRL  has  supported  projects  in  MMR,  Honey  Lake,  Ft.  Bliss,  etc.  If  the 
USACE  can  simply  request  AFRL  to  perfonn  a  clearance  somewhere,  why  or 
how  could  the  private  sector  compete  with  that?  Eventually  autonomy  in 
operations  will  be  attained  by  AFRL. 

AFRL  performs  R&D  and  conducts  experiments  in  Autonomous  Range 
Clearance.  AFRL  does  not  compete  with  industry  for  range  clearance. 

Will  the  teams  and  their  competitors  be  publicly  announced? 

We  expect  to  announce  the  competitors. 

Taking  into  consideration  the  current  political  and  economic  environment -What 
impact  or  risk  will  the  announcement  of  t  he  adm  inist  r  ation’s  decision  t  o 
send  an  additional  30  thousand  troops  into  Afghanistan  have  on  this  effort,  i.e., 
schedule  slip;  loss  of  identified  funding,  etc.,  keeping  in  mind  that  this  troop 
increase  is  not  in  the  current  budget.  So  the  military  and  Congress  will  have  to 
find  money  to  support  and  typically  look  to  sourcing  through  cuts  to  programs 
either  ongoing  or  new  programs/efforts. 

OSD  and  DA  G-3  have  committed  as  long  as  we  have  competitors. 

Will  the  potential  Government  shutdown  and  /  or  continuing  resolution  situation 
affect  the  Robotics  Range  Clearance  efforts?  Are  there  contingency  plans  if  this 
occurs? 

Yes  a  potential  Government  shutdown  could  affect  the  R2C2.  All  the  necessary 
funds  have  been  obligated  for  the  preparation,  management  and  operation  of  the 
event,  but  a  lengthy  or  untimely  shutdown  could  impact  the  government 
personnel's  ability  to  run  the  competition.  A  contingency  plan  has  not  been 
developed  for  this  unlikely  scenario,  but  the  competition  would  be  conducted  at 
a  later  date.  This  may  include  an  alternate  location  and  possibly  reduced  scope 
based  on  the  situation. 
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2.2.2.  Competition  Rules 

2.2.2. 1  Are  we  considering  GPS  denied  areas  as  a  consideration  in  the  scoring? 

ANS:  Robust  solutions  are  desired  but  performance  will  be  measured,  not  GPS 

denied  specifically. 

2.2. 2. 2  Interactions  per  acre  or  per  time? 

ANS:  Interactions  will  be  measured  during  the  entire  operation.  No  association 

between  interactions  and  acreage  has  been  established.  See  R2C2  Rules  and 
Metrics  document. 

2.2. 2. 3  In  the  business  there  is  enhanced  teleops — automated.  Will  teleop  be  prohibited? 

ANS:  Tele-op  is  not  prohibited. 

2.2. 2. 4  Will  we  pick  a  site  that  matches  what  a  team  intends  to  do? 

ANS:  There  will  be  a  common  geophysical  mapping  range  for  all  competitors  in  that 

category.  Other  sites  will  be  assigned  based  on  the  categories  the  competitor 
team  intends  to  perform  in. 

2. 2.2. 5  On  vegetation  -  how  are  you  to  rate  vegetation  removal  by  size  of  equipment  and 
level  of  autonomy?  An  expensive  high  production  compared  to  several 
inexpensive  low  capability  units? 

ANS:  It  will  be  evaluated  on  productivity,  capability,  and  effectiveness. 

2.2. 2. 6  How  will  competitors  be  judged?  Example,  under  vegetation  clearance  is 
clearance  rate  more  or  less  important  than  amount  of  material  removed.  For 
example,  if  I  can  leave  large  trees  in  place,  I  may  be  able  to  do  it  quicker. 

ANS:  Competitors  perfonnance  will  be  judged  on  the  end  state  achieved  in  relation  to 

the  perfonnance  requirement. 

2.2. 2. 7  You  have  stated  that  each  competitor  will  have  a  different  area  to  clear  at  Camp 
Guernsey.  Will  these  areas  be  mapped  ahead  of  time  so  that  you  know  what  is 
there?  Otherwise,  how  will  you  evaluate  vehicle  perfonnance  at  that  location? 

ANS:  Yes.  Sites  will  be  surveyed  ahead  of  time. 

2.2. 2. 8  Will  foreign  clearance  companies  be  allowed  to  participate  as  a  sub-contractor  to 
a  US  prime  contractor? 

ANS:  Yes,  pending  legal  review  and  approval. 
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2. 2.2. 9  Is  the  sensor  suite  for  mapping  and  marking  specified?  Can  you  describe  it,  or 
what  are  the  characteristics  of  ordnance,  i.e.,  metallic,  etc.? 


ANS:  No,  not  specified.  Ordnance  items  of  interest  will  include  20mm  to  155mm  and 

are  metallic. 

2.2.2. 10  Why  is  marking  with  flags  separate  from  excavation  for  an  autonomous  robotic 
system? 

ANS:  Marking  is  no  longer  a  part  of  the  competition.  See  current  R2C2  Rules  and 

Metrics. 

2.2.2. 1 1  How  large  an  area  for  vegetation  clearance?  What  is  comms  range?  LOS? 
Frequencies?  Will  there  be  comms  deconfliction?  What  will  be  the  slope? 

ANS:  The  exact  size  of  the  area  is  TBD.  The  communication  range  will  depend  on  the 

communication  strategy  created  by  the  competitor  team.  No  personnel  will  be 
allowed  within  the  exclusion  zone  of  870M.  Line  of  site  is  not  ensured. 
Frequencies  will  be  requested  by  the  competitor  and  de-conflicted  by 
competition  oversight  primarily  through  geographic  separation  and  scheduling. 
Based  on  Camp  Guernsey’s  spectrum,  bands  and  power  limits  will  be  suggested. 
Slope  see  R2C2  Rules  and  Metrics  doc. 

2.2.2. 12  Can  we  change  the  3000  feet  exclusion  to  say  3000  feet  from  the  operational 
equipment  as  opposed  to  3000  feet  from  the  perimeter? 


ANS:  Yes.  See  current  Rules  and  Metrics  doc.  This  rule  now  states  870  M  from  the 

robot. 

2.2.2. 13  How  many  shifts  per  day? 

ANS:  The  competitor  team  will  establish  how  many  shifts  per  day.  R2C2  is  not 

specifying.  Teams  will  address  this  in  their  operational  safety  plan.  Any  shift 
duration  deemed  to  be  unsafe  will  be  addressed. 

2.2.2. 14  Will  we  be  given  a  chance  to  give  them  a  return  home  function  if  comms  are 
lost? 

ANS:  See  R2C2  Rules  and  Metrics  doc.  The  -Loss  of  Communications  Stop!  rule 

requires  that  the  system  halt  and  cease  operations.  Communications  can  be 
regained  and  operations  resumed.  If  a  team  proposes  a  retro-traverse,  please  tell 
us  and  we  will  consider  it. 

2.2.2.15  Define  range  perimeter? 

ANS:  For  the  purposes  of  the  competition  the  range  perimeter  is  the  work  space 

boundary  for  the  performance  task.  See  R2C2  Rules  and  Metrics  doc. 
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2.2.2.16 

ANS: 

2.2.2.17 

ANS: 

2.2.2.18 

ANS: 

2.2.2.19 
ANS: 

2.2.2.20 
ANS: 
2.2.2.21 
ANS: 

2.2.2.22 

ANS: 

ANS: 

ANS: 

2.2.2.23 

ANS: 

2.2.2.24 


JGRE  JAUS  compatibility? 

It  is  not  required. 

How  does  the  ESCIP  focus  on  UXO  classification  apply  to  the  geophysical 
mapping  component  of  the  prize  competition? 

Competition  and  rules  focus  on  deployment  of  the  sensor,  not 
evaluation/classification  of  the  data. 

Driver  is  cheaper,  safer  (?).  If  someone  is  to  bring  a  teleop  system  will  they  be 
able  to  compete. 

Yes.  Tele-op  can  compete. 

Does  monitoring  equate  to  human  interaction? 

No 

Will  duration  of  interaction  be  measured? 

Yes 

Define  payload?  Driving  or  buck  ops  is  payload?  (?) 

There  is  no  specific  reference  to  payload  in  the  R2C2  Rules  and  Metrics  doc. 
This  question  is  unclear  but  seems  to  ask  if  there  is  a  difference  in  interaction 
weighting  between  driving  and  bucket  operations.  It  is  the  intent  of  the 
competition  to  capture  all  human  interactions  and  level  of  interaction  with  the 
systems. 

Would  a  pre-range  walk  count: 

as  a  human  interaction  -  No 
as  a  function  of  time  -  Yes 

what  if  you  gathered  data  on  the  walk  around  and  uploaded  the  data?  Uploading 
is  an  interaction 

Will  competitors  be  allowed  to  perform  a  range  walk  prior  to  employing  their 
system  for  any/all  categories? 

Yes 

How  many  acres  will  be  clear  cut  for  vegetation  removal? 
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ANS: 

The  scope  of  the  competition,  to  include  the  size  of  the  ranges,  will  be 
determined  after  all  Letters  of  Intent  have  been  received  and  the  R2C2  makes  a 
determination  on  the  number  of  viable  competitor  teams  relative  to  the  available 
resources. 

2.2.2.25 

For  competition,  what  will  be  the  mass  required  to  be  lifted? 

ANS: 

The  maximum  mass  required  to  be  lifted  will  be  the  mass  of  the  largest  piece  of 
simulated  munitions  of  concern  representing  a  155  mm  artillery  round. 

2.2.2.26 

How  fast  do  tasks  need  to  be  completed?  What  is  the  time  limit  on  the 
competition? 

ANS: 

The  scope  of  the  competition,  to  include  the  duration  of  each  event,  will  be 
determined  after  all  Letters  of  Intent  have  been  received  and  the  R2C2  makes  a 
determination  on  the  number  of  viable  competitor  teams  relative  to  the  available 
resources.  The  intent  of  the  R2C2  is  that  events  will  last  from  a  few  days  to  no 
more  than  a  week. 

2.2.2.27 

If  they  fly  a  manned  vehicle,  how  does  this  affect  the  scoring? 

ANS: 

Given  that  the  competitor  team  receives  permission  from  Camp  Guernsey  and 
follows  all  applicable  regulations  and  procedures,  time  and  man-hours  to  fly  any 
mission  will  count  against  the  team’s  event  score. 

2.2.2.28 

For  all  of  the  areas,  can  the  organizers  provide  additional  detail  regarding  the 
density  of  seeding  for  the  areas?  Are  there  metrics  or  guidelines  available  that 
will  give  guidance  as  to  the  number  of  objects  per  acre  or  some  other 
quantitative  measure? 

ANS: 

Each  area  will  be  from  2-5  acres  in  approximate  size.  The  areas  will  simulate 
artillery  and  mortar  targets  with  up  to  1 82  kgs  per  acre  of  UXO  debris  with  an 
approximate  70/30  surface  to  sub-surface  ratio.  The  types  and  sizes  of  the  UXO 
items  are  described  in  the  ISO  description  document  on  the  R2C2  SharePoint 
site.  The  vegetation  clearance  area  will  be  composed  of  a  mix  of  tall  grass,  large 
bushes,  and  trees  with  up  to  25  trees  per  acre  (up  to  36  cm  trunk  dia.) 

2.2.2.29 

If  the  seeding  of  the  areas  is  expected  to  be  dense,  will  the  organizers  be 
providing  magnetic  sifters  similar  to  the  ones  used  at  Camp  Edwards  for  the 
competition,  or  are  teams  expected  to  purchase  sifters  as  part  of  the  surface 
clearance  event? 

ANS: 

No,  teams  must  provide  all  of  their  own  equipment. 

2.2.2.30 

Will  there  be  any  canopy  issues  that  may  deny  GPS  reception? 

ANS: 

There  are  no  anticipated  or  planned  areas  in  the  competition  where  canopy 
should  interfere  with  GPS  reception.  For  the  geophysical  mapping,  surface 
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clearance,  and  sub-  surface  clearance  events  there  should  be  no  canopy  issues. 
For  the  vegetation  clearance  event,  no  canopy  issues  are  anticipated  due  to  the 
density  of  the  tree  cover.  But  the  competition  organizers  cannot  rule  this  out. 
Teams  should  plan  accordingly. 

2.2.2.31 

Will  the  competition  areas  for  geophysical  mapping  and  both  clearance  tasks  be 
cleared  from  obstacles  such  as  boulders,  trees,  etc.?  Will  there  be  large  enough 
obstacles  that  we  would  need  OD/OA  (boulders,  etc.)  for  a  vehicle  such  as  an 
ATV  to  traverse? 

ANS: 

Each  of  the  ranges  will  have  obstacles  constructed  of  railroad  ties  that 
competitor  vehicles  and  equipment  are  expected  to  avoid.  There  is  no  explicit 
competition  requirement  for  traversing  these  obstacles. 

2.2.2.32  What  data  will  be  provided  on  the  sites  and  when  (i.e.,  terrain,  site  location, 


maps,  photos,  collection  area  location  for  debris  and  seeded  items,  command 
area  location,  ingress/egress  path  from  pit  area  to  task  areas,  etc)? 

ANS: 

Data,  such  as  the  types  mentioned  above,  will  be  provided  to  all  competitors  two 
weeks  prior  to  the  start  of  the  competition. 

2.2.2.33 

What  will  be  the  distance  between  command  location  and  the  task  area? 

ANS: 

Competitors  should  be  prepared  to  operate  vehicles  more  than  870  meters  from 
the  command  location. 

2.2.2.34 

For  the  subsurface  clearance  task,  will  the  dig  coordinates  for  50  buried  ISOs 
contain  the  buried  depth  of  the  seeded  item? 

ANS: 

Competitors  will  be  provided  dig  coordinates  and  measures  of  signal  strength  for 
each  of  the  seeded  metallic  items.  The  current  version  of  the  rules  does  not 
specify  the  actual  number  of  ISOs. 

2.2.2.35 

Can  you  provide  any  information  on  the  dispersed  density  of  the  seeded  items? 
For  example,  will  the  seeded  items  be  equally  dispersed  throughout  the  task  area 
or  will  there  be  just  a  few  higher  density  areas? 

ANS: 

For  the  sub-surface  clearance  event,  competitors  will  be  provided  a  dig  list  of 
seeded  metallic  items  (ISOs)  to  remove.  For  the  other  events,  no  more 
information  is  provided  about  seeded  metallic  item  density  or  location. 

2.2.2.36 

Will  any  of  the  tasks  be  done  over  the  same  ground?  Or  will  each  task  that  a 
team  will  perfonn  be  done  over  new  ground? 

ANS: 

The  only  shared  range  for  all  competitors  is  the  geophysical  mapping  range. 
Competitors  will  be  provided  a  range  area  that  only  they  will  operate  on  for  each 
of  the  other  events. 
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2.2.2.37 

ANS: 

2.2.2.38 

ANS: 

2.2.2.39 

ANS: 

2.2.2.40 

ANS: 

2.2.2.41 

ANS: 


Can  you  provide  information  on  the  containment  area?  Will  there  be  a  bin  that 
the  seeded  items  have  to  be  placed  in  or  just  an  area  on  the  ground? 

All  containment  areas  will  be  areas  marked  on  the  ground. 


Can  you  provide  more  information  on  the  penalties  for  surface  grade  and  surface 
damage? 

The  rules  document  specifies  the  penalties  for  changes  in  surface  grade  and 
surface  damage. 

Can  you  explain  and  give  more  detail  on  the  survey  speed  requirement?  What  is 
the  point  of  maintaining  constant  speed  for  geophysical  mapping? 

The  competition  assumes  that  electromagnetic  induction  sensors  are  typically 
most  effective  in  geophysical  mapping  when  accelerations  of  the  sensor  are 
minimized. 

The  system  shall  have  an  Emergency  Stop  (E-Stop).  The  system  must  halt 
within  15  meters  and  cease  all  equipment  operations  when  the  E-Stop  is 
initiated.  The  E  stop  must  be  remotely  activated.  Question:  to  us  all  "cease  all 
equipment  operation"  means  -  vehicle  stops  and  is  not  moving,  -  nothing  on  the 
vehicle  is  moving.  However,  it  could  be  interpreted  to  mean  "engine  is  stopped". 
Could  you  clarify?  Specifically,  do  we  need  to  turn  of  the  engine  when  the  E_stop 
is  engaged?  Note  that  this  would  mean  that  we  have  to  manually  restart  the 
engine. 

For  the  e-stop  the  best  case  is  for  the  engine  to  be  stopped.  You  are  allowed  to 
go  down  range  to  restart  it.  It  just  cost  the  time  to  do  it.  It  can  be  left  on,  but  you 
must  show  how  your  design  would  prevent  unintended  movements. 

Is  it  acceptable  if  the  flashing  light  is  on  continuously  when  the  system  is 
powered  up,  or  can  it  only  be  on  when  the  system  is  moving  (and  10  seconds 
prior  to  this?) 

per  the  rules  document  page  1 1 .  "Each  vehicle  shall  be  equipped  with  a  warning 
light  that  is  activated  according  to  the  state  of  the  E-Stop  system. 

Each  vehicle  shall  display  one  or  more  flashing  amber  warning  lights,  the 
combination  of  which  results  in  visibility  360  degrees  azimuthally  around  the 
vehicle.  The  warning  light(s)  shall  operate  when  the  vehicle  is  in  E-stop  RUN 
mode.  The  vehicle  may  not  commence  movement  until  the  warning  light(s)  has 
been  in  operation  for  5  seconds.  The  warning  light(s)  shall  comply  with  SAE 
Class  1  standards  for  warning  lights  and  shall  not  produce  light(s)  than  can  be 
confused  with  those  of  public  safety  vehicles  such  as  law  enforcement,  fire,  or 
ambulance."  Strictly  following  the  rules  your  light  should  only  operate  when  the 
vehicle  is  in  the  e-stop  "RUN"  mode.  The  best  way  to  accomplish  this  is  to  wire 
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the  light  power  into  the  e-stop  circuit.  The  required  delay  before  movement  is  5 
seconds  not  10. 

2.2.2.42  Warning  Devices  -  We  interpret  that  you  want  the  warning  light  to  be  flashing  - 
at  least  5  seconds  prior  to  the  vehicle  is  supposed  to  start  moving  (it  sounds  like 
20  seconds  would  be  fine  as  well)  -while  the  vehicle  is  moving.  When  the 
vehicle  is  not  moving,  and  is  not  supposed  to  be  moving,  the  light  should  be  off. 
When  the  motor  is  running,  but  the  vehicle  is  not  supposed  to  be  moving,  the 
light  should  be  off.  Can  you  confirm  that  this  is  correct? 

ANS:  For  the  warning  light,  the  rule  is  self  explanatory.  If  you  have  it  on  when  the 

system  is  off  that  is  your  choice,  but  it  is  not  very  useful  to  warn  of  impending 
motion. 


2.2.2.43  Loss  of  communications  stop  -  The  system  shall  automatically  halt  and  cease 
operations  if  communications  with  the  system  are  lost  or  interrupted  for  a 
maximum  of  2  seconds  and  may  travel  no  farther  than  30  meters.  The  question 
here  is  similar  to  the  one  for  the  E-stop.  We  assume  that  loss  of  communication 
should  result  in  the  vehicle  stopping.  However,  we  would  leave  the  engine 
running  (so  that  if  comms  was  lost  and  then  resumed  the  vehicle  could  continue 
the  mission).  Could  you  confirm  that  this  assumption  is  correct  (ie  loss  of 
communication  stop  allows  for  vehicle  engine  running  as  long  as  vehicle  is 
stopped?) 

ANS:  You’re  correct  in  your  interpretation  of  the  loss  of  communication  rule.  Once 

again  the  rule  is  self  explanatory. 

2.2.2.44  No  freewheel  -  The  systems  shall  not  be  capable  of  motion  when  stopped  or 
un-powered.  The  way  we  interpret  this  is  that  when  the  vehicle  is  stopped  or  the 
engine  is  off  it  cannot  freewheel.  Our  stop  solution  (which  applies  the  brake)  is 
powered  from  a  battery.  We  assume  that  this  is  allowed?  So,  un-powered 
purely  applies  to  the  engine,  and  not  to  the  control  system? 

ANS:  For  the  no  freewheel  rule,  unpowered  means  unpowered.  So  the  use  of  battery 

power  to  hold  the  brake  does  not  confonn  to  this  rule.  There  are  many  methods 
available  to  apply  brakes  or  stop  motion  when  no  power  is  available. 

2.2.2.45  Are  the  Army  Corps  digital  geophysical  mapping  QC  requirements  required  for 
this  competition? 

ANS:  No.  Each  competitor  is  solely  responsible  for  the  quality  of  their  geophysical 

data. 


2.2.3.  Competition  Procedures 

2.2.3. 1  Can  you  publish  the  reports  documenting  the  work,  production,  lessons  lear  ned, 
what  worked  and  what  doesn’t ,  strategies  used,  et  c.  ? 
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ANS: 

2.23.2 
ANS: 

2.23.3 
ANS: 

2.23.4 
ANS: 

2.23.5 
ANS: 

2.23.6 
ANS: 

2.23.7 
ANS: 

2.23.8 

ANS: 

2.23.9 
ANS: 
2.2.3.10 
ANS: 


There  is  a  public  release  approved  document  available  online  at 
www.roboticrangeclearance.com  that  captures  prior  work  AFRL  has  done  in  this 
domain. 

Will  competitors  have  access  to  area  prior  to  run? 

Yes. 

What  will  be  the  slope? 

Terrain  will  vary  from  relatively  flat  to  sloping  up  to  45%  with  interspersed 
ravines,  ditches,  rocks. 

Communications  de-confliction? 

Will  be  included  in  the  packet.  Will  cover  test,  entire  band,  wattage  will  be 
constant,  will  need  advance  notice  of  requirements.  De-confliction  shall  be 
accomplished  through  geographic  separation  and  through  scheduling. 

Will  communications  be  line  of  sight  or  indirect? 

Line  of  site  is  not  ensured. 

Foreign  national  visitor  approval? 

Foreign  national  visitors  must  comply  with  Camp  Guernsey  site  physical 
security  requirements  which  will  include  escort  provided  by  competitor  team. 

Will  competitors  have  to  prepare  and  submit  work  plans  and  safety  plans? 

Yes. 

EM  385-1-97  and  other  DoD  publications  state  that  when  mechanical  means  are 
utilized  for  UXO  removal,  the  maximum  fragmentation  distance  will  be 
observed  as  an  exclusion  zone,  i.e.,  155mm  HE  projectile  +  2,577  feet.  Will  an 
exclusion  zone  be  established  for  personnel  standoff  distance? 

Yes.  For  the  purposes  of  the  competition  the  exclusion  zone  is  870M  from  the 
system. 

What  is  the  expected  range  of  (LOS/NLOS)  operation? 

LOS  is  not  ensured.  The  competition  does  not  specify  your  comms  solution. 
Limit  on  the  number  of  competitors? 

No. 
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2.2.3.11 

How  many  competitors  to  move  forward?  2-3? 

ANS: 

OSD  will  decide. 

2.2.3.12 

Will  Competition  documents  be  available  in  the  public  domain?  For  example, 
Competition  rules,  Qualification  results,  Evaluation  criteria. 

ANS: 

Yes. 

2.2.3.13 

Can  we  get  an  example  GIS.DQM.SMP  (sp?)  file  package  from  Guernsey  to 
review  and  consider  in  planning  and  design? 

ANS: 

Yes.  This  has  been  requested  based  on  the  anticipated  competition  site. 

2.2.3.14 

Spectator  area? 

ANS:  Maybe.  The  R2C2  plans  to  establish  a  spectator  area  if  the  resources  are 

available  to  safely  establish  one. 


2.2.3.15 

ANS: 

Sponsors;  are  you  going  to  make  this  a  media  event  so  sponsors  will  care? 

Yes. 

2.2.3.16 

ANS: 

Will  frequencies  in  5-6  GHz  be  available? 

Camp  Guernsey  cannot  make  any  frequencies  -available!  as  the  Wyoming 
National  Guard  is  not  a  communications  regulatory  authority.  There  are  no  users 
on  Camp  Guernsey  in  that  band.  A  brief  license  search  with  the  FCC  turned  up 
no  local  licenses  in  that  band  except  at  5.9452  GHz.  Assuming  your  operating 
compliance  with  FCC  regulations  there  is  nothing  that  would  restrict  your  use  of 
frequencies  in  the  5-6  GHz  range. 

2.2.3.17 

Can  you  provide  a  section  in  the  FAQ  that  records  questions  asked  but  yet  to  be 
answered  so  we  know  the  question  has  been  received? 

ANS: 

Yes.  If  R2C2  feels  that  the  question  is  a  duplicate  of  one  already  received  and 
answered  no  attempt  will  be  made  to  capture  the  new  question.  In  short,  we 
expect  that  a  competitor  has  reviewed  the  latest  version  of  the  Q&A  to 
determine  if  their  question  has  already  been  answered  before  asking  a  new 
question. 

2.2.3.18 

How  will  we  know  the  teams  have  lost  comms? 

ANS: 

We  will  simulate  loss  of  comms  to  test  and  confirm  the  system  is  compliant  with 
rules. 

2.2.3.19 

Applies  to  loss  of  either  E-stop  comm  and  (?)? 

ANS:  - 

Loss  of  communication!  is  the  loss  of  all  communication  with  system. 
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2.2.3.20 

ANS: 

Fire  extinguishers  in  each  vehicle? 

We  are  not  requiring  fire  suppression  systems  on  the  competitor  systems.  Fire 
extinguishers  will  be  available  at  Camp  Guernsey  for  team  and  pit  areas.  Teams 
are  encouraged  to  bring  their  own  extinguishers.  The  potential  of  a  fire  will  be 
addressed  by  teams  in  their  safety  plan. 

2.2.3.21 

Halon  required  on  the  vehicle? 

ANS: 

No.  See  above. 

2.2.3.22 

What  are  the  qualifications  of  the  safety  officer?  Will  they  be  required  to  be 

UXO  (?)... 

ANS: 

The  team  will  be  responsible  for  detennining  qualifications.  UXO  teams  are  not 
required. 

2.2.3.23 

If  remotely  3000  feet  away,  how  will  you  know  the  boundary? 

ANS: 

We  will  provide  GIS. 

2.2.3.24 

ANS: 

Will  accurate  mapping  and  contour  be  provided? 

Teams  will  be  given  a  GIS  data  package  for  their  competition  site.  R2C2  intends 
to  detennine  the  accuracy  of  the  mapping  data  via  ground  truth  survey  and  make 
that  data  available  to  competitors. 

2.2.3.25 

ANS: 

Can  they  fly  UAVs  first? 

The  anticipated  competition  sites  are  within  the  restricted  airspace  R-7001  at 
Camp  Guernsey.  There  is  no  restriction  in  the  rules  regarding  UAS  operations 
but  competitors  will  be  required  to  comply  with  UAS  regulations.  Please  provide 
the  R2C2  oversight  with  more  infonnation  as  soon  as  possible  to  discuss  the  use 
of  UASs  by  teams.  It  should  be  noted  that  there  are  no  prepared  airfield 
capabilities  within  R-7001. 

2.2.3.26 

Civilian  manned  aircraft  ...  ( ?)  . 

ANS: 

The  anticipated  competition  sites  are  within  the  restricted  airspace  R-7001  at 
Camp  Guernsey.  There  is  no  restriction  in  the  rules  regarding  civilian  manned 
aircraft  operations  but  competitors  will  be  required  to  comply  with  Camp 
Guernsey  flight  regulations.  Please  provide  the  R2C2  oversight  with  more 
information  as  soon  as  possible  to  discuss  the  use  of  manned  aircraft  by  teams. 

2.2.3.27 

If  all  systems  to  be  used  in  the  competition  are  fully  functional  and  in 
operational  mode  at  active  installations,  do  we  have  to  participate  in  the 
qualification  trials? 

ANS: 

The  Qualification  Trials  have  been  cancelled  and  replaced  by  an  In-Process 
Review.  See  R2C2  Rules  and  Metrics  doc. 
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2.2.3.28  How  do  you  expect  robot  operations  to  begin  at  3000  feet? 

ANS:  See  R2C2  Rules  and  Metrics  document. 

2.2.3.29  What  about  hard  cord  connection? 

ANS:  There  is  nothing  in  the  R2C2  Rules  and  Metrics  Document  that  precludes  cable 

control  operations. 


2.2.3.30  Could  you  provide  more  explanation  on  what  the  competitive  field  generally 
may  look  like? 

ANS:  Yes.  When  the  sample  GIS  data  package  has  been  prepared  we  will  include 

photographs  and  general  site  description. 

2. 2. 3. 3 1  How  accurate  is  GIS,  slope,  etc.? 

ANS:  Undetermined.  Camp  Guernsey  has  GIS  data  that  has  not  been  -ground  truthedll 

in  the  competition  site  yet. 

2.2.3.32  Range  footprint  for  DGM  site? 

ANS:  Will  be  survey  on  total  station  and  that  data  will  be  available. 

2.2.3.33  What  altitude  does  the  installation  own? 

ANS:  Camp  Guernsey’s  restricted  airspace  R-7001  dimensions  are  in  the  public 

domain.  The  altitude  that  the  installation  -owns!  depends  on  what  tier  of  R-7001 
is  activated. 

2.2.3.34  Will  packet  of  lessons  learned  be  made  available  for  review?  When?  Will  the 
package  be  posted  to  the  web  site? 

ANS:  Yes.  They  are  available  at  www.roboticrangeclearance.com 

2.2.3.35  The  statement  was  made  that  the  decision  to  allow  an  inspection  of  the 
performance  has  not  been  made.  My  suggestion  would  be  to  allow  a  site-walk 
consistent  with  an  inspection  of  the  range  area  prior  to  a  procurement.  The 
benefits  include  visual  recognition  of  hazard  areas  or  avoidance  features.  Would 
a  site-walk  of  this  nature  be  allowed? 

ANS:  Yes. 

2.2.3.36  Who  is  liable  for  a  fire? 

ANS:  The  competitors  will  be  responsible  to  meet  the  guidelines  they  have  established 

in  their  safety  plan.  If  those  have  been  met  the  installation  will  be  liable. 
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2.2.3.37  Not  to  change  profile.  Does  this  include  the  sub-surface  task? 

ANS:  Yes.  The  tolerances  are  provided  in  the  R2C2  Rules  and  Metrics  doc. 

2.2.3.38  Will  the  geophysical  mapping  area  be  located  in  open  terrain  where  GPS  signal 
is  attainable? 

ANS:  Yes. 


2.2.3.39  How  does  the  300  foot  exclusion  compare  to  the  3000  feet  exclusion? 

ANS:  Removed.  See  latest  R2C2  Rules  and  Metrics  doc 

2.2.3.40  Will  you  expect  the  system  to  look  for  and  avoid  UXOs  during  brush  cutting 
operations? 

ANS:  Not  at  this  time. 

2.2.3.41  How  will  we  be  judged?  Time,  production,  quality? 

ANS:  See  Rules  and  Metrics  Doc.  The  three  major  judging  categories  are  Task 

Performance,  Level  of  Human  Interaction,  and  Man-hours. 

2.2.3.42  Are  unit  operating  costs  going  to  be  evaluated  as  part  of  the  prize  competition? 

ANS:  No. 

2.2.3.43  Are  the  organizers  aware  of  the  AUVSI  Unmanned  Systems  North  America 
Conference  that  will  be  held  Aug.  16-19  in  Washington,  DC?  This  is  the  largest 
unmanned  systems  conference  in  the  nation,  and  a  majority  of  the  teams  will 
want  to  participate  in  this  conference.  Will  the  organizers  consider  moving  the 
dates  of  the  R2C2  to  accommodate  this  conference? 

ANS:  Yes  we  are  aware  of  AUVSI’s  schedule.  The  selection  and  use  of  Camp 

Guernsey  for  the  competition  allows  a  small  window  to  accommodate  the 
competition  (R2C2).  The  R2C2  must  be  conducted  after  the  majority  of  the 
summer  military  exercises  at  Camp  Guernsey  (June  and  July)  and  before 
inclement  weather  sets  in  (possible  in  September). 

With  the  current  field  of  four  (4)  competitors,  the  R2C2  should  run  from 
Monday  8  August  to  Sunday  14  August  (if  there  are  no  weather  days)  allowing 
team  personnel  time  to  travel  to  AUVSI  if  they  desire.  It  is  possible  to  begin 
R2C2  on  Thursday  4  August  (end  on  Thursday  1 1  August),  by  reducing  the 
onsite  practice  time  to  3  days.  All  teams  would  have  to  agree  to  this.  (Teams  are 
not  permitted  to  conduct  practice  runs  during  other  teams’  scheduled 
competition  events). 
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2.2.3.44 

ANS: 

2.2.3.45 

ANS: 

2.2.3.46 

ANS: 

2.2.3.47 

ANS: 

2.2.3.48 

ANS: 


2.2.3.49 


For  the  testing  effort  and  for  the  actual  competition  we  will  have  groundpower 
available  in  our  pit  area? 

Yes 

For  the  week  of  august  1st,  we  will  have  access  to  a  test  area.  This  test  area 
will  not  have  seeded  items,  but  will  be  similar  in  terrain  to  the  competition  area? 

Yes 


One  thing  I  am  not  sure  about  is  whether  in  the  test  area  there  will/will  not  be 
obstacles  similar  to  those  in  the  competition  area.  Could  you  let  me  know? 

Yes 

We  know  that  we  will  get  the  coordinates  of  the  obstacles  (I  guess  8  comer 
points).  Would  we  also  get  the  corner  points  of  the  obstacles  on  the  test  range? 

Yes 

Would  you  be  able  to  tell  us  in  which  coordinate  system  and  with  reference  to 
which  datum  these  points  will  be  provided?  Note  that  there  is  always  some 
difference  between  NAD83  and  WAG84,  and  it  is  thus  important  for  us  to  know 
what  coordinates  these  would  be  in  so  we  can  work  with  similar  coordinates  in 
our  off  site  validation  efforts. 

Answer  part  1:  Coordinate  System  is  in  UTMAVGS  84/UTM  zone  13N.  UTM 
zone  13N  is  the  datum  reference.  Answer  part  2:  Because  of  the  inherent  nature 
of  GPS  the  JTEC  base  station  is  about  +/-0.5  to  +/-0.25  meters  accurate  in 
reference  to  the  UTM  GLOBAL  coordinate  system  so  we  will  provide  additional 
information  in  the  form  of  localization  points  such  that  the  LOCAL  coordinate 
system  is  has  centimeter  level  accuracy.  Three  Localization  points  (in  the 
competitor  information  packet)  will  be  provided  for  each  field  (many  fields 
share  localization  points).  These  localization  points  will  be  provided  both 
physically  (physical  monument  locations)  and  digitally  (in  the  provided  SHP 
file).  The  shape  files  for  the  site  will  be  supplied  two  weeks  prior  to  the 
competition  as  discussed.  We  will  provide  this  information  no  later  than  close  of 
business  on  Friday  22  July,  2011. 

I  was  wondering  if  you  can  provide  some  additional  details  wrt  industry  day  on 
the  1 1th.  Specifically,  will  it  take  place  on  the  competition  site,  or  at  a  nearby 
practice  site?  What  facilities  (if  any)  will  there  be  for  us  to  use  (e.g.  tables/tents, 
power)?  In  case  of  inclement  weather,  will  we  be  inside  somewhere? 
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ANS:  Please  refer  to  the  email  sent  on  June  30  regarding  the  details  for  the  visitor  day. 

This  will  take  place  at  the  competition  site.  The  excerpts  that  describe  the 
activities  are:  "As  a  reminder  for  the  media  and  visitor  day  on  1 1  August,  this 
will  be  a  publicly  attended  event  with  news  media  reporting.  The  sample 
itinerary  for  the  media  day  is  attached.  Note  -  the  details  and  times  may  change 
slightly  but  this  is  the  general  flow.  Please  understand,  the  presentation  time 
allotted  to  your  team  is  at  your  discretion  for  what  you  want  to  present  and 
demonstrate,  including  what  information  you  wish  to  divulge.  My 
recommendation  is  that  you  provide  an  introduction  and  overview  of  your  team 
and  organizations  capabilities,  the  technology  you  have  chosen  to  use  (your 
choice  on  detail),  demonstrate  your  system(s)  in  the  practice  area  (as  much  or 
little  as  you  want),  and  then  tour  the  systems  or  have  a  Q&A  session  or  just 
network  -  entirely  up  to  you.  We  are  providing  the  forum  for  you  to  present 
your  team  and  capabilities  to  the  media  and  visitors.  We  will  provide  tent  space 
and  practice  ranges  for  the  briefs  and  demos,  but  once  again  it's  your  time  to 
present  what  you  choose.  Please  let  us  know  any  resources  or  special  needs  that 
you  may  have  to  enable  your  presentations."  Once  again  please  let  us  know 
what  if  any  resources  you  require  and  we  will  try  to  accommodate  them  and 
arrange  it  prior  to  your  departing  for  Wyoming. 

2.2.3.50  We  know  that  we  will  have  to  hand  you  the  raw  data  within  30  minutes  of 

completion.  This  will  not  be  an  issue.  We  also  know  that  we  will  have  to  provide 
merged  data  (ie  data  with  position  information)  within  48  hours.  This  will  not  be 
an  issue.  We  are  not  clear  about  whether  we  are  expected  to  do  anything  else. 
Specifically,  there  are  four  following  additional  tasks:-  noise  level 
determination,  -  sensor  coverage  assessment,  -  anomaly  location  accuracy,  - 
survey  speed  assessment.  We  have  some  questions  both  about  WHO  does  this, 
and  HOW  this  is  done  (see  below).  Noise  level  determination  -  this  will  be 
determined  by  the  R2C2  Competition  Team.  ->  is  that  us,  or  is  that  you? 
(from  what  I  can  see,  this  is  the  only  time  the  tenn  "  R2C2  Competition  Team" 
is  used).  Sensor  coverage  assessment  -  score  will  be  determined  by  the  percent 
of  the  range  covered. 

Ql:  I  assume  this  is  the  percent  of  the  range  minus  the  obstacles? 

Q2:  how  will  you  determine  coverage?  The  requirement  is  0.5  m  line  spacing. 
However,  at  some  moment  there  was  the  comment  "we  want  you  to  go  right  up 
to  the  obstacle".  Could  you  clarify  this? 

Q3:  who  determines  this?  Do  you  detennine  this?  Do  we  determine  it  as  well? 
Anomaly  Location  Accuracy  -  is  this  us,  or  is  this  you?  If  this  is  us,  do  we 
provide  you  with  the  standard  Oasis  output?  When  is  this  due?  Survey  speed  -  is 
this  us,  or  is  this  you?  Does  survey  speed  only  get  calculated  for  when  we  are 
collecting  data  or  also  for  turns 
outside  the  survey  data?  When  is  this  due? 

ANS:  According  to  the  rules,  competitors  are  only  required  to  deliver  raw  data  within 

30  minutes  and  merged  data  within  48  hours.  The  other  assessments  are  made 
from  the  competitor-provided  data  by  the  R2C2  judges. 
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2.2.4.  Competition  Procedures  (GFE) 

2.2.4. 1  More  on  GFE  -  Geophysical  sensors  as  GFE? 

ANS:  No. 

2.2. 4. 2  ARTS  GFE  with  attachments? 

ANS:  ARTS  only. 

2. 2.4. 3  ARTS  equipment  and  robotic  support  GFE? 

ANS:  ICD  and  maintenance  support. 

2. 2.4.4  Excavator,  dozer,  etc.  GFE? 

ANS:  No. 

2. 2.4. 5  Do  you  have  an  idea  of  how  many  ARTS  vehicles  would  be  available  as  GFE? 

ANS:  We  said  9  at  Industry  Day.  It  is  highly  unlikely  that  anyone  will  get  one  for  the 

whole  time  much  less  two.  It  will  depend  on  the  level  of  interest. 

2. 2. 4. 6  Superday? 

ANS:  There  will  be  a  media  event  scheduled  for  the  competition. 

2. 2.4. 7  Flail  or  range  master  will  a  10-15%  growth  rate  (?) 

ANS:  If  the  growth  rate  is  outside  of  the  surface  grade  change  limit  of  15cm  there  will 

be  penalties  assessed. 

2.2. 4. 8  The  OSD  is  mandating  at  least  two  mobilizations  of  the  equipment  to  the  test 
site.  First  for  the  qualification  and  the  second  for  the  competition.  Given  the 
cost  of  equipment  mobilization  and  since  the  primary  purpose  of  the 
qualification  is  to  verify  the  equipment  meets  the  competition  published  safety 
requirements  will  OSD  consider  the  following:  1 .  Have  the  evaluation  team 
travel  to  the  competitors  site  to  view  equipment  and  perform  the  qualification  or: 
2.  Allow  competitors  to  skip  the  qualification  phase  and  work  at  their  risk  up  till 
the  competition.  At  which  time  the  competition  committee  will  inspect 
equipment  in  two  phases.  Phase  one  is  on  the  trailer.  If  the  equipment  does  not 
meet  requirements  it  is  not  unloaded  until  meets  standard  or  rejected  completely. 
Phase  2  is  operational  test.  Again  if  the  equipment  does  not  meet  requirements 
and  cannot  be  promptly  modified  it  is  disqualified  without  further 
testing/evaluation.  It  will  be  the  compet  it  or  ’s  responsibility  to  promptly 
remove  the  equipment  from  the  test  site? 
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ANS:  R2C2  has  considered  this  and  has  cancelled  the  Qualification  Trials  and  replaced 

them  with  an  In-Process  Review  at  the  competitor’s  site.  See  R2C2  Rules  and 
Metrics. 

2.2. 4. 9  If  a  company  is  and  has  known  capabilities  are  they  required  to  physically  attend 
the  qualification  round  or  can  they  submit  a  video  of  systems? 

ANS:  The  Qualification  Trials  have  been  cancelled  and  replaced  by  and  In-Process 

Review. 

2.2.4. 10  Will  the  seeded  range  that  you  perfonn  DGM  work? 

ANS:  The  seeded  geophysical  mapping  range  will  utilize  buried  industry  standard 

objects.  See  R2C2  rules  and  Metrics  doc  for  more  information. 

2.2.4. 1 1  Would  that  be  the  same  that  we'll  perform  removal  action  on? 

ANS:  The  geophysical  mapping  range  that  is  to  be  used  for  the  geophysical  mapping 

task  score  is  not  the  same  as  the  surface  or  subsurface  removal  sites. 

2.3.  Unanswered  Questions 

2.3.1.  Questions  related  to  the  IDIQ 

2. 3. 1.1  Is  there  a  guarantee  of  funding  for  IDIQ? 

ANS: 
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Appendix  K:  2009 — Robotics  Prize  Competition  Kick-off  Meeting  Briefing 


Joint  Ground  Robotics 
Enterprise 


Robotics  Prize  Competition 
Kick-Off  Meeting 

Mr.  Brian  Skibba 
AFRL/RXQF  Robotics  Group 
Competition  Technical  Director 


Agenda 


0800-0820  hours  -  Opening/Welcome  (Mr.  Tony  Melita,  Mr.  Scott  Miller) 

0820-0930  -  Competition  Overview  (Mr.  Brian  Skibba) 

IDIQ  Overview  (Mr.  Plyler  McManus) 

0930-0950  -  Break 

0950-1050  -  Range  Clearance  Process  Overview  (Mr.  Andy  Schwartz) 
1050-1135  —  AFRL  Robotics  Clearance  Overview  (Mr.  Brian  Skibba) 


1135-1300-  Lunch 


1300  -  1330-Camp  Guernsey  Brief  (Mr.  Brian  Beyer) 
1330-  1430-Q  and  A 
1430-1600  -  Networking 
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Robotic  Prize 
Competition  Goal 

•  Advance  the  state  of  the  art  in  robotics 
range  clearance  technologies  to  more 
safely  and  efficiently  clear  munitions  ranges 
of  debris  and  unexploded  ordnance 

•  The  ultimate  aim  is  to  be  able  to  clear  the 
millions  of  acres  currently  encumbered  with 
spent  training  rounds  and  munitions'  debris 
to  place  the  land  back  into  productive  use. 


Robotic  Prize 

Competition  Technical  Goal 

•  To  advance  the  state  of  the  art  from  remote 
controlled  tele-operation  to  semi/full  autonomous 
robotic  operation 

•  But,  this  is  not  about  sensor  or  platform 
development ...  it  is  an  integration  and  robotic 
algorithm/behavior  challenge 


There  already  existing  a  wide  variety  of  machines  and 
implements  to  perform  all  the  required  tasks  for  UXO  range 
clearance.  What  is  needed  are  autonomous  controls  to 
perform  the  task  more  safely  and  efficiently. 
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“I  have  pleaded  with  DoD  labs  to  conduct  prize  competitions. 
There  are  many  DoD  programs  with  difficult  technological 
challenges  that  could  be  addressed  through  the  use  of  a  prize 
competition  model.  Additionally,  the  fringe  benefits  of  such  a 
competition  are  incredible  for  DoD,  small  businesses,  industry, 
young  people  and  the  Nation.  I  again  challenge  DoD  labs  and 
program  offices  to  identify  challenges  and  conduct  new  prize 
competitions.” 


John  J.  Young 
USD(AT&L) 


Congress  has  given  us  the  authority,  let’s  use  it! 

Public  Law  110-36  Section  2374a.  Prizes  for 
Advanced  Technology  Achievements 

5 


Competition  Concept 


•  In  order  to  effect  a  smooth  transition  from  technology  development 
to  system  procurement,  the  competition  will  be  predicated  on  key 
decision  points. 

1 .  Determine  the  level  of  industry  interest  in  participating  in  a  competition  as  well 
as  the  ability  to  develop,  then  produce  quantities  of  robotic  range  clearance 
systems. 

Complete  as  evidenced  by  the  participation  -  18  RFI  responses,  50+  organizations  at  kick-off 

2.  The  competition  event  will  be  preceded  by  a  qualification  requirement. 
Competitors  must  first  demonstrate  that  they  can  enter  a  viable  robotic  system 
capable  of  safely  executing  tasks  inherent  in  clearing  ranges. 

3.  If  a  sufficient  quantity  of  competitors  qualify  to  participate  in  the  competition,  the 
DoD  will  proceed  with  the  competition  and  award  $2M  in  cash  prizes  and  also 
pursue  a  procurement  contract  to  purchase  robotic  range  clearance  systems. 
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•  The  competition  is  will  be  an  assessment  of  the  competitors’  system 
performance  against  a  specified  set  of  metrics  that  satisfy  the 
operational  requirements  for  range  clearance. 

-  These  metrics  are  being  finalized  by  the  competition  committees. 


•  The  systems  will  not  to  be  compared  against  each  other.  The 
winner(s)  will  be  determined  by  achieving  the  best  performance  to 
the  metrics  for  the  designated  tasks. 


•  The  system  performance  may  be  characterized  based  on  multiple 
executions  of  each  task  with  the  score  being  the  average  of  each 
trial  value. 

-  The  specific  rules  and  metrics  are  being  finalized  by  the  competition 
committees. 


Notional  Prize 
Competition  Parameters 

•  Prizes  awarded  based  on  measurable  performance,  e.g.: 

•  Vegetation  Clearance 

-  For  a  given  area,  automated  system  to  clear  all  vegetation  from 
area  (up  to  14”  trees  and  other  parameters) 

•  Sensor  employment  for  Detection,  Mapping  &  Marking  : 

-  Given  a  sensor  suite,  provide  an  automated  system  that  can  map 
surface/subsurface  clutter. 

•  Surface  Clearance: 

-  For  given  previously  mapped  grid  -  speed  at  which  surface  is 
cleared,  %  surface  clutter  cleared  (with  allowances  for  variable 
density) 

•  Subsurface  Clearance: 

-  For  given  previously  mapped  grid  -  speed  at  which  subsurface  is 
cleared,  %  subsurface  clutter  cleared  (with  allowances  for  variable 
density) 
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Proposed  Cash  Prize  Distribution 


•  A  Cash  Prize  of  $1 M  will  be  awarded  to  the  Overall  Competition 
Winner. 


•  A  Cash  Prize  of  $250K  will  be  awarded  to  the  best  performance  in 
each  of  the  Performance  Areas. 


•  Competitors  can  choose  to  compete  in  one  or  more  Performance 
Areas. 


•  The  Overall  Winner  will  be  that  competitor  who  scores  the  best 
across  multiple  Performance  Areas. 

•  It  will  be  possible  for  the  Overall  Winner  of  the  $1 M  cash  prize  to 
also  win  the  Performance  Area  cash  Prizes. 


9 


Notional  Prize  Scope 


•  autonomous  vegetation  clearance  ($250K) 

•  autonomous  sensor  employment  for  detection,  mapping 
&  marking  ($250K) 

•  autonomous  surface  debris  clearance  ($250K) 

•  autonomous  sub-surface  UXO  clearance  ($250K) 

•  End  Goal:  Area  Autonomously  Cleared  of  UXO  ($1 M) 


Focus  is  on  increasing  safety  and  operational  effectiveness 
via  robotics  automation  as  well  as  reduce  time  and  cost 

10 
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Prize  Competition  Parameters 


•  16  months  for  competitors  to  develop  and  qualify  systems 

•  Range  location  and  preparation  conducted  simultaneously  with 
vehicle  development 

•  Qualification  and  competition  events  under  control  of  range 
owners  and  DoD  UXO  personnel  ...  not  competitors 

•  Competitors  wishing  to  compete  agree  to  conditions  of 
competition  being  not  entirely  identical ...  performance 
assessed  based  on  “normalized”  metrics  ...  this  is  a 
competition  not  a  test 

•  Prize  awarded  based  on  best  performance  not  on  perfect 
clearance  of  live  range  acreage 


SAFETY  SAFETY  SAFETY!!! 
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Roles  and  Responsibilities1 


•  OSD/JGRE 

-  Provides  competition  design,  management,  and  cash  prize  funding 

-  Provides  oversight  to  competition  planning,  design,  coordination  with 
industry,  allocation  and  execution  of  funds 

•  US  Air  Force  Research  Laboratory 

-  Serves  as  the  T echnical  Manager  to  ensure  appropriate  autonomy 
technical  considerations  in  design,  management,  and  execution  the 
qualification  trails  and  competition 

•  US  Army  Corps  of  Engineers: 

-  Serves  as  the  Operational  Manager  to  ensure  appropriate  operational 
considerations  in  design,  management,  and  execution  of  the 
qualification  trails  and  competition  itself 

•  US  Army  G3/5/7 

-  Serves  as  the  voice  of  the  customer  with  respect  to  range  clearance 
concerns 

-  Provides  IDIQ  contract  funding 

•  US  Army  Environmental  Center 

-  Provides  operational  support  to  design,  manage,  and  execute 
qualification  trails  and  competition  trials 

12 
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Notional  Schedule  FY10-FY11 

Oct  Nov  Dec  Jan  Feb  Mar  Apl  May  Jun  Jly  Aug  Spt  Oct 

Finalize  Competition  Planning  A 

A  Request  fc 

Site  Surveys  A 

Assess  RFI  Results  A  A 

Proceed  w /  Competitic 

Competition 

Desian  (®  Camp  A 

A 

Information 

A 

i  ^ 

Guernsey 

13 
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Oct  Nov 


A 


Notional  Schedule  FY10-FY11 

2011  (FY111 

Dec  Jan  Feb  Mar  Apl  May  Jun  Jly  Aug  Spt  Oct 

_ UGV  Refinement 

Sufficient  Qualifying  Systems  to  proceed 
with  Competition/Acquisition? 

Issue  RFP 


A. 


A 


Proposal  Preparation 
Prize 


A 


Proposals 
Received  by  Govt 


Proposal  Evaluation  &  Source  Selection 


ZA_A 


Close-out 


A* 


Contract  Negotiation  &  Award  occurs  in  FY12 


15  15 


Competition  Contract 
Concept 

•  The  concept  for  the  competition  is  to  gather  data  and  advance 
technologies  to  enable  a  competitive  acquisition  award  by  the  Army 
Corp  of  Engineers  for  Robotic  Range  Clearance  Equipment  and 
services 


•  Possible  scenarios: 

-  No  competitor(s)  comes  close  to  objectives  -  thus  no  award  and  no 
follow-on  contract.  OSD  works  to  program  future  funding  to  continue 
technology  development  to  provide  autonomous  capability  for  next 
rotation 


-  Competitor(s)  proves  ability  to  automate  clearance  tasks  but  not  at 
sufficient  maturity  level  to  immediately  field  -  possible  follow-on 
acquisition  contract  would  require  appropriate  level  of  development 
leading  to  a  procurement  contract 

-  Competitor(s)  demonstrates  successful  "fieldable"  solution  -  follow  on 
competitive  acquisition  contract  could  be  initiated  immediately. 

16 
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Competition  Concept 


•  To  determine  as  early  as  possible  the  likelihood  of  the  3  possible  outcomes 
qualifying  trials  will  be  scheduled  to  ensure  a  meaningful  final  competition. 

•  Two  types  of  awards  will  be  offered,  a  cash  prize  and  a  follow-on  contract 
award: 

-  A  cash  prize  of  $2M  will  be  awarded  against  the  main  range  clearance 
tasks. 

-  In  addition  to  the  cash  prizes,  competitors  will  also  be  evaluated  for 
award  of  a  follow-on  contract  to  either  develop  a  fieldable  system 
(outcome  2)  or  the  desired  IDIQ  contract  for  deployment  of  the  new 
capability  to  conduct  actual  clearance  of  live  ranges  (outcome  3). 

•  In  order  to  achieve  the  desired  outcome  of  the  IDIQ  contract,  competitors 
will  be  asked  to  submit  formal  Proposals  in  response  to  a  formal  Request 
for  Proposal  (RFP). 

-  Proposals  will  be  evaluated  in  accordance  with  a  basis  of  selection 
outlined  in  the  RFP  which  will  include  proposal  evaluation  factors  along 
with  the  competitors’  resulting  performance  in  the  final  physical  cash 
prize  competition. 

Possible  that  a  competitor  may  win  a  cash  prize  but  not  be 
selected  for  the  follow-on  contract  award  and  also  possible  that 
a  competitor  may  win  both  a  cash  prize  and  follow-on  contract 
award.  17 


Key  Considerations 
and  Resources 


•  Government  can  provide  up  to  16  All-Purpose  Remote 
Transport  Systems  (ARTS)  and  the  Robot  Intelligence 
Kernel  (RIK) 

•  Competitors  can  choose  to  use  Government 
Furnished  Equipment  (GFE)  or  not 

•  Cooperative  Research  And  Development  Agreements 
(CRADA)  or  Other  Transaction  Agreements  (OTA) 
provide  legally  binding  agreements  for  liability/ 
accountability 

•  Need  a  process  for  adjudication  if  more  competitors 
intent  to  use  ARTS  than  the  number  of  platforms 
available  (first  come/first  served;  lottery;  etc.) 

18 
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All-Purpose  Remote  Transport 
Systems  (ARTS) 


Background:  The  ARTS  is  a  fielded,  low-cost,  survivable  robotics 
platform  (8100  lbs.)  capable  of  remote  operations  in  various  mission 
profiles.  The  system  can  remotely  employ  an  array  of  tools  and 
attachments  to  detect,  assess,  and  render  safe  large  improvised 
explosive  devices  (lEDs)  and  large-vehicle  bombs  as  well  as  clear 
unexploded  ordnance  (UXO)  from  prepared  areas.  In  addition,  the 
system  employs  a  variety  of  advanced  navigation,  control,  and  sensing 
systems. 


Performance: 

Endurance:  6-8  hours 
Control:  Radio-1.5  mile  radius 

(minimum;  line  of  sight) 


19 


Robot  Intelligence  Kernel 


The  RIK  provides  the  following  core  behaviors: 

•  Mapping  and  localization,  obstacle  avoidance,  and  path  planning 
for  autonomous  navigation 

•  Recognize  changes  in  map  and  video 

•  Levels  of  autonomy 

•  Adaptive  communication 

Operational  parameters  or  remote  robotics  operations: 

•  Enables  operations  in  communication-  and  GPS-denied 
environments 

•  Novices  using  the  Intelligence  Kernel  out-perform  experts  using 
teleop 

•  Remote  operations  extend  for  hundreds  of  miles  instead  of 
meters 

20 
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CRADA 


Definition  -  The  Cooperative  Research  and  Development  Agreement 
(CRADA)  is  a  contractual  mechanism  used  for  technology  transfer.  It 
is  an  Agreement  between  a  prospective  industry  partner  and  the 
government  in  which  research  and  development  knowledge,  facilities, 
resources,  and/or  capabilities  are  used  in  cooperation  to  the  mutual 
benefit  of  both  private  and  government  sectors. 

•  With  regard  to  resources,  Federal  law  prohibits  the  payment  of 
government  funds  to  any  non-federal  partner 

•  While  a  CRADA  is  a  contract,  it  is  not  a  part  of  the  government’s 
acquisition  and  procurement  process.  Federal  acquisition 
regulations  do  not  apply. 


21 


Key  Dates  and 
information 

•  Industry  Day: 

-  10  December  2009,  Tyndall  AFB,  Panama  City,  FL 

-  Formal  announcement  with  rules,  metrics,  and  team 
commitment 

-  Demonstrations  of  robotic  range  clearance  technologies, 
Q&A  and  hands  on  time  with  robots 

•  Competition  website: 

http://roboticranqeclearance.com/ 

•  Official  announcements  through  DoD  RFI  process 


Competition  website  will  be  main  focal  point  for  all 
announcements  and  distribution  of  data. 
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Summary 


•  DoD  is  looking  for  the  Robotics  Prize  Competition  to: 

-  advance  the  state  of  the  art  in  robotics  range  clearance 
technologies 

-  foster  opportunity  for  COTS  procurement  for  Robotic  Range 
Clearance 

-  Provide  the  best  balance  of  efficiency  and  innovation  in  robotic 
technology  development 


•  Hope  we  see  you  at  the  Competition! 
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LIST  OF  SYMBOLS,  ABBREVIATIONS,  AND  ACRONYMS 


AFB 

Air  Force  Base 

AFRL 

Air  Force  Research  Lab 

AHUM 

Assitant  Human  Interaction  Team  Member 

ARA 

Applied  Research  Associates 

ARTS 

All-Purpose  Remote  Transport  System 

AUVSI 

Association  for  Unmanned  Vehicles  Systems  International 

CIMAR 

Center  for  Intelligenct  Machines  and  Robotics 

DoD 

Department  of  Defense 

dvr 

digital  video  recorder 

EOD 

explosives  ordnance  disposal 

E-stop 

emergency  stop 

GPS 

Global  Positioning  System 

HI 

human  interaction 

IDIQ 

indefinite  delivery  indefinite  quantitiy 

IOI 

items  of  interest 

IPR 

in  process  reviews 

IPT 

Integrated  Product  Team 

ISOs 

Industry  Standard  Objects 

JGRE 

Joint  Ground  Robotics  Enterprise 

JTEC 

Joint  Training  and  Experimentation  Center 

LOI 

Letter  of  Intent 

OCU 

operator  control  unit 

OUSD/AT&L 

Office  of  the  Undersecretary  of  Defense,  Acquistion,  Technology,  and 
Logistics 

PA 

public  announcement 

PHITM 

Primary  Human  Interaction  Team  Member 

Q&A 

questions  and  answers 

QC 

quality  control 

R&D 

research  and  development 

R2C2 

Robotics  Range  Clearance  Competition 

RF 

radio  frequency 

RFI 

Request  for  Information 

RXQE 

Materials  and  Manufacturing  Directorate,  Airbase  Technologies  Division' 
Airbase  Engineering  Development  Branch 

SAE 

Society  of  Automotive  Engineers 

SPAWAR 

Spacea  and  Naval  Warfare  Systems  Command 

TARDEC 

Tank  Automotive  Research  Development  and  Engineering  Center 

UF 

University  of  Florida 

USA 

United  States  Army 

USACOE 

United  States  Army’s  Corps  of  Engineers 

USAF 

United  States  Air  Force 

USN 

United  States  Navy 

UTM 

Universal  Transverse  Mercator 

UXO 

unexploded  ordnance 

VAC 

volts  alternating  current 
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